Page: full programme sections

index.html

@@ -146,7 +146,8 @@
     color: var(--text);
     transition: border-color 0.2s;
   }
-  .space:hover { border-color: var(--accent); }
+  a.space:hover { border-color: var(--accent); }
+  .space .desc { max-width: 560px; }
   .space .name { font-weight: 500; font-size: 0.95rem; }
   .space .desc { font-size: 0.8rem; color: var(--muted); }
   .space .arrow { color: var(--muted); font-size: 1.1rem; }
@@ -225,8 +226,10 @@
     <h2>Where We Are</h2>
     <ul class="research-list">
       <li><strong>v2 against its base.</strong> 12 benchmarks: 2 improved, 2 regressed, 8 with no measurable difference.</li>
-      <li><strong>Defect found.</strong> In the merge that produced v1, inherited by v2. Corrected, and being re-benchmarked.</li>
-      <li><strong>Not yet shown.</strong> A release that improves on its base on every axis. Each iteration is built to recover the regressions and raise the rest.</li>
+      <li><strong>Improved.</strong> GSM8K under HELM's protocol: 87.0 against 83.3 (+3.7, calibrated). MMLU under HELM's protocol: +0.47 (calibrated).</li>
+      <li><strong>Regressed.</strong> MATH Level 5: -5.74. MMLU-Pro: -1.89.</li>
+      <li><strong>Defect found.</strong> In the merge that produced v1, inherited by v2. Corrected. The corrected v1 measures at its base model's level on MMLU-Pro (TIGER-Lab).</li>
+      <li><strong>Not yet shown.</strong> The end state: a release that improves on its base on every axis. Each iteration is built to recover the regressions and raise the rest.</li>
     </ul>
   </div>
 
@@ -241,40 +244,117 @@
 
   <div class="section">
     <h2>The Goal</h2>
-    <p><strong>Top placements in class, across the public leaderboards.</strong> 7B first, then each larger size class and other model families. Each release controls the trade-offs between skills, and iteration then raises every axis, until no public fine-tune improves any axis further.</p>
+    <p><strong>Top placements in class, across the public leaderboards.</strong></p>
+    <ul class="research-list">
+      <li><strong>Scope.</strong> 7B first. Then each larger size class and other model families.</li>
+      <li><strong>Method.</strong> Each release controls the trade-offs between skills. Iteration then raises every axis: reasoning, mathematics, code, instruction following and knowledge.</li>
+      <li><strong>End state.</strong> Improvement on every axis, with no statistically significant regression against the base model.</li>
+      <li><strong>Done.</strong> A class is complete when no public fine-tune improves any axis further.</li>
+    </ul>
   </div>
 
   <div class="section">
     <h2>The Process</h2>
     <ul class="research-list">
       <li><strong>Start</strong> from a strong open base model in the class</li>
-      <li><strong>Discover</strong> the compatible fine-tunes the community has published</li>
+      <li><strong>Discover</strong> the compatible fine-tunes the community has published for it</li>
       <li><strong>Assimilate</strong> them into one model</li>
-      <li><strong>Measure</strong> every axis against the base, on identical items and public protocols</li>
-      <li><strong>Release</strong> with every gain and every regression published</li>
-      <li><strong>Iterate</strong> until nothing more improves, then carry the process to the next class</li>
+      <li><strong>Measure</strong> every axis against the base model, on identical items and public protocols</li>
+      <li><strong>Release</strong> with every gain and every regression published, against the base model and the previous release</li>
+      <li><strong>Iterate</strong> to recover the regressions and raise the weaker axes, until nothing more improves. Then the next class</li>
     </ul>
+    <p style="margin-top:1rem">The construction method is proprietary. The evaluation is public.</p>
   </div>
 
   <div class="section">
-    <h2>Research Releases</h2>
-    <p>Releases are published under this organisation from converge v3 onwards. The research that led to them is on the founder's page.</p>
+    <h2>Releases</h2>
+    <p>Published under this organisation from converge v3.</p>
     <div class="spaces">
-      <a href="https://huggingface.co/Optitransfer/Qwen2.5-7B-Instruct-converge-collective-v2" class="space">
+      <div class="space">
         <div>
-          <div class="name">converge v2</div>
-          <div class="desc">The current release, with a 12-benchmark validation board</div>
+          <div class="name">converge v3</div>
+          <div class="desc">Next iteration. Built to recover v2's regressions and raise the remaining axes</div>
         </div>
-        <span class="arrow">&rarr;</span>
-      </a>
+      </div>
+      <div class="space">
+        <div>
+          <div class="name">converge v4</div>
+          <div class="desc">Planned. The iteration after v3</div>
+        </div>
+      </div>
+      <div class="space">
+        <div>
+          <div class="name">Larger classes</div>
+          <div class="desc">Planned. The same process, at each larger size class and other model families</div>
+        </div>
+      </div>
+    </div>
+  </div>
+
+  <div class="section">
+    <h2>Research</h2>
+    <p>The research behind the releases is on the founder's page, <a href="https://huggingface.co/Optitransfer" style="color:var(--accent)">@Optitransfer</a>.</p>
+    <div class="spaces">
       <a href="https://huggingface.co/Optitransfer/Qwen2.5-7B-Instruct-converge-collective-v1" class="space">
         <div>
           <div class="name">converge v1</div>
-          <div class="desc">The first 7B release; see the correction notice on its card</div>
+          <div class="desc">Public. First 7B experiment, on Qwen2.5-7B-Instruct. Carries the defect; see the notice on its card</div>
         </div>
         <span class="arrow">&rarr;</span>
       </a>
+      <a href="https://huggingface.co/Optitransfer/Qwen2.5-7B-Instruct-converge-collective-v2" class="space">
+        <div>
+          <div class="name">converge v2</div>
+          <div class="desc">Public. Built on v1, with the 12-benchmark board above. Inherits the defect</div>
+        </div>
+        <span class="arrow">&rarr;</span>
+      </a>
+      <div class="space">
+        <div>
+          <div class="name">converge v1, corrected</div>
+          <div class="desc">Built. Full board in progress</div>
+        </div>
+      </div>
+      <div class="space">
+        <div>
+          <div class="name">converge v2, corrected</div>
+          <div class="desc">Built. Full board in progress</div>
+        </div>
+      </div>
     </div>
+    <p style="margin-top:1rem">The corrected models' boards are published when complete, whichever way they fall.</p>
+  </div>
+
+  <div class="section">
+    <h2>Evidence Standard</h2>
+    <ul class="research-list">
+      <li><strong>Paired.</strong> Every score is the model minus its base on the same items, with a 95% confidence interval and an exact significance test</li>
+      <li><strong>Regressions reported.</strong> A card lists what went down as prominently as what went up</li>
+      <li><strong>Calibration labelled.</strong> Calibrated means the harness first reproduced the base model's published score. Every result is marked either way</li>
+      <li><strong>Public protocols.</strong> HELM-protocol GSM8K and MMLU, MMLU-Pro (TIGER-Lab), ZeroEval, EvalPlus HumanEval+ and MBPP+, MATH, AIME, ARC, IFEval</li>
+      <li><strong>Corrected in public.</strong> When a defect is found, the affected cards say so first</li>
+    </ul>
+  </div>
+
+  <div class="section">
+    <h2>Open Questions</h2>
+    <ul class="research-list">
+      <li>Can trade-offs be controlled, and every axis then raised through iteration, across a full board?</li>
+      <li>Does it beat ensembling, routing and best-of-n sampling at matched compute?</li>
+      <li>Does it carry to larger classes and other model families?</li>
+      <li>Does it hold on calibration, hallucination and instruction following?</li>
+    </ul>
+    <p style="margin-top:1rem">Each answer is published as it is measured.</p>
+  </div>
+
+  <div class="section">
+    <h2>Who It Is For</h2>
+    <ul class="research-list">
+      <li><strong>Open community.</strong> A stronger open model per class, with the evidence to check it</li>
+      <li><strong>Regulated industries.</strong> Healthcare and finance teams whose fine-tunes cannot leave their boundary, and who must show how each model change was made</li>
+      <li><strong>National programmes.</strong> Capability that grows by contribution, without dependence on a few vendors</li>
+    </ul>
+    <p style="margin-top:1rem"><strong>Open model, paid guarantees.</strong> The public model stays open. The planned commercial offering is private consolidation of an organisation's own fine-tunes, inside its boundary, with the same measurement and evidence.</p>
   </div>
 
   <div class="section">
@@ -341,8 +421,13 @@
     </div>
   </div>
 
+  <div class="section">
+    <h2>Team</h2>
+    <p><strong>Ryan Gillespie</strong>, founder. Fifteen years of verification and validation on safety-critical systems: railway signalling (Crossrail, ETCS), medical devices taken to CE mark (Roche) and aviation (Lilium). Converge applies that discipline to model weights.</p>
+  </div>
+
   <footer class="footer">
-    Ryan Gillespie, founder &middot; Switzerland &middot; <a href="mailto:data@optitransfer.ch">data@optitransfer.ch</a>
+    Switzerland &middot; <a href="mailto:data@optitransfer.ch">data@optitransfer.ch</a>
   </footer>
 
 </div>