Add OFT All 140k to Easy, Hard and Plus with verified completion counts

README.md

@@ -57,4 +57,4 @@ All original scores (94/400, 222/400, 294/400), native videos, evidence ZIPs, se
 
 The Easy, Hard and Plus panels include `StarVLA/Qwen3-VL-OFT-RoboTwin2-All` (140k; 27,500 clean/randomized training demonstrations; 50-action chunks). Each panel preserves its established instruction and native evaluation protocol. Completion counts are reported separately from successes. All prior episodes remain unchanged.
 
-[Campaign evidence](https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/0a35b0e10a79afeaab65912a123b5448559fb7cd/research/oft-all140k-20261103/summary.json).
+[Campaign evidence](https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/c833b470484d05a4afda77fbb6bb8cf24057d4d9/research/oft-all140k-20261103/summary.json).

app.js

@@ -119,7 +119,7 @@ function opusPilotProgress(){
  return `<section class="panel" style="padding:18px 24px;margin-bottom:24px" aria-label="Claude Code campaign"><strong>Claude Code · Opus 5.5 / high</strong><p>${number(n.finished||0)} / ${number(c.planned_episodes)} episodes completed · ${number(n.running||0)} running · ${number(n.queued||0)} pending${n.needs_review?' · '+number(n.needs_review)+' under review':''}${n.interrupted?' · '+number(n.interrupted)+' interrupted':''}.</p><p class="source-note">${c.quota_waiting?'Waiting for model-service quota recovery · ':''}${number(c.published_episodes)} episode records published · <a href="${esc(c.status_url)}" target="_blank" rel="noopener">Results and protocol ↗</a></p></section>`;
 }
 function methodology() {
-  return `<details class="panel methodology" id="methodology"><summary><span><span class="eyebrow">Reading the benchmark</span><strong>Protocol, metrics & provenance</strong></span><span class="summary-plus">+</span></summary><div class="method-grid"><div><h3>One selected episode</h3><p>Each standalone checkpoint: 50 tasks, one episode each, official seed group 0 with native expert admission. VLAct Clean 50K, VLAct All 100K and the original Base 40K episodes are all retained. Kinex: the retained three-task seed-0 pilot. Kinex + VLA: the October 30 VLAct Clean 50k run on those same three tasks and seed 0; previous valid attempts remain archived.</p></div><div><h3>Native success</h3><p>Success rate is passes / evaluated tasks, pooled over the selected episodes. Pending tasks do not count as failures. Results follow the original native verifier.</p></div><div><h3>Native control budget</h3><p>RLE-Bench task03 · RoboTwin 2.0 · ALOHA AgileX. Standalone uses official take_action and task-specific limits. Kinex uses rosbridge control frames. Initializations and physical durations are not matched.</p></div><div><h3>Policy & agent</h3><p>Non-VLAct All 140k: StarVLA/Qwen3-VL-OFT-RoboTwin2-All, step 140,000, trained on 27,500 clean/randomized demonstrations. It uses three 224 × 224 images and full 50-action chunks. The existing Easy instruction protocol is retained; expert admission matched Base scene seeds on 45/50 tasks.</p><p>Standalone VLAct Clean: StarVLA/VLAct_Qwen3OFT_Robotwin_Finetune, step 50,000, RoboTwin Clean/Base training mixture. It uses the same released 32-action, image-bucket and wrap-joint contract as VLAct All. Official expert admission selected the same scene seed as All in 45/50 tasks. Each checkpoint has one admitted episode per task; training mixture and training duration differ, so these results do not isolate a causal mechanism. </p><p>Standalone VLAct All: StarVLA/VLAct_Qwen3OFT_Robotwin_all_Finetune, step 100,000, 32-action chunks and checkpoint-specific image buckets / wrap decoding. Official expert admission selected the same scene seed as Base 40k in 43 of 50 tasks; this is not a fully paired comparison. Standalone Base 40k retains StarVLA/Qwen3-VL-OFT-Robotwin2, step 40,000. Kinex + VLA now uses VLAct Clean 50k, with 32 predicted targets and its checkpoint-specific image buckets and wrap decoder. Kinex uses GPT-6 Astra / high. Kinex chooses whether to use VLA, writes its control loop, and chooses language, arbitrary JSON conditions and optional formatting. Additional conditions enter the frozen model through language. This interface supports richer inputs; effectiveness is a separate experimental question. The rerun returns untimed joint waypoints, executed by Kinex trajectories or agent-authored code. Encoder changes and progress monitors are not used.</p></div><div><h3>Token accounting</h3><p>${usageDefinition()} API-equivalent cost uses the recorded Pi session estimates; account charges and GPU costs are not reported.</p></div></div><div class="method-links"><a href="${esc(evidenceURL('research/oft-all140k-20261103/protocol.json'))}">All 140k protocol & checkpoint ↗</a><a href="${esc(resultURL('sources.json'))}" target="_blank">Data sources ↗</a><a href="history.html#overview">Previous rosbridge pilot ↗</a><a href="${esc(evidenceURL('research/vlact-clean-20261029/results.json'))}">VLAct Clean results ↗</a><a href="${esc(evidenceURL("research/vlact-all-20261029/results.json"))}">VLAct All results ↗</a><a href="${esc(evidenceURL("official/results.json"))}">Base 40k results ↗</a><button class="text-button" data-export="all">Download comparison CSV ↓</button></div></details>`;
+  return `<details class="panel methodology" id="methodology"><summary><span><span class="eyebrow">Reading the benchmark</span><strong>Protocol, metrics & provenance</strong></span><span class="summary-plus">+</span></summary><div class="method-grid"><div><h3>One selected episode</h3><p>Each standalone checkpoint: 50 tasks, one episode each, official seed group 0 with native expert admission. VLAct Clean 50K, VLAct All 100K and the original Base 40K episodes are all retained. Kinex: the retained three-task seed-0 pilot. Kinex + VLA: the October 30 VLAct Clean 50k run on those same three tasks and seed 0; previous valid attempts remain archived.</p></div><div><h3>Native success</h3><p>Success rate is passes / evaluated tasks, pooled over the selected episodes. Pending tasks do not count as failures. Results follow the original native verifier.</p></div><div><h3>Native control budget</h3><p>RLE-Bench task03 · RoboTwin 2.0 · ALOHA AgileX. Standalone uses official take_action and task-specific limits. Kinex uses rosbridge control frames. Initializations and physical durations are not matched.</p></div><div><h3>Policy & agent</h3><p>Non-VLAct All 140k: StarVLA/Qwen3-VL-OFT-RoboTwin2-All, step 140,000, trained on 27,500 clean/randomized demonstrations. It uses three 224 × 224 images and full 50-action chunks. The existing Easy instruction protocol is retained; expert admission matched Base scene seeds on 45/50 tasks.</p><p>Standalone VLAct Clean: StarVLA/VLAct_Qwen3OFT_Robotwin_Finetune, step 50,000, RoboTwin Clean/Base training mixture. It uses the same released 32-action, image-bucket and wrap-joint contract as VLAct All. Official expert admission selected the same scene seed as All in 45/50 tasks. Each checkpoint has one admitted episode per task; training mixture and training duration differ, so these results do not isolate a causal mechanism. </p><p>Standalone VLAct All: StarVLA/VLAct_Qwen3OFT_Robotwin_all_Finetune, step 100,000, 32-action chunks and checkpoint-specific image buckets / wrap decoding. Official expert admission selected the same scene seed as Base 40k in 43 of 50 tasks; this is not a fully paired comparison. Standalone Base 40k retains StarVLA/Qwen3-VL-OFT-Robotwin2, step 40,000. Kinex + VLA now uses VLAct Clean 50k, with 32 predicted targets and its checkpoint-specific image buckets and wrap decoder. Kinex uses GPT-6 Astra / high. Kinex chooses whether to use VLA, writes its control loop, and chooses language, arbitrary JSON conditions and optional formatting. Additional conditions enter the frozen model through language. This interface supports richer inputs; effectiveness is a separate experimental question. The rerun returns untimed joint waypoints, executed by Kinex trajectories or agent-authored code. Encoder changes and progress monitors are not used.</p></div><div><h3>Token accounting</h3><p>${usageDefinition()} API-equivalent cost uses the recorded Pi session estimates; account charges and GPU costs are not reported.</p></div></div><div class="method-links"><a href="${esc(evidenceURL('research/oft-all140k-20261103/REPORT.md'))}">All 140k results & validation ↗</a><a href="${esc(evidenceURL('research/oft-all140k-20261103/protocol.json'))}">All 140k protocol & checkpoint ↗</a><a href="${esc(resultURL('sources.json'))}" target="_blank">Data sources ↗</a><a href="history.html#overview">Previous rosbridge pilot ↗</a><a href="${esc(evidenceURL('research/vlact-clean-20261029/results.json'))}">VLAct Clean results ↗</a><a href="${esc(evidenceURL("research/vlact-all-20261029/results.json"))}">VLAct All results ↗</a><a href="${esc(evidenceURL("official/results.json"))}">Base 40k results ↗</a><button class="text-button" data-export="all">Download comparison CSV ↓</button></div></details>`;
 }
 function taskBrowser(variant=null) {
   return `<section class="task-browser" id="tasks"><div class="section-heading bare"><div><div class="eyebrow">${variant?'Episode index':'Browse the benchmark'}</div><h2>Explore tasks <span class="heading-count">${plannedTasks(variant).length}</span></h2></div><button class="button secondary" data-export="${variant||'all'}">Export CSV ↓</button></div><div class="panel"><div class="filters"><label class="search-field"><span aria-hidden="true">⌕</span><input id="task-search" type="search" aria-label="Search tasks" placeholder="Search task name, instruction or ID" value="${esc(filters.query)}"></label><select id="task-family" aria-label="Environment"><option value="">All environments</option>${data.families.map(f=>`<option value="${f.id}" ${filters.family===f.id?'selected':''}>${f.name}</option>`).join('')}</select>${variant?'':`<select id="task-variant" aria-label="Agent for outcome filter">${data.variants.map(v=>`<option value="${v.id}" ${filters.variant===v.id?'selected':''}>${v.label||v.agent} · ${v.model}</option>`).join('')}</select>`}<select id="task-status" aria-label="Outcome"><option value="">All outcomes</option>${['pass','fail','pending'].map(s=>`<option value="${s}" ${filters.status===s?'selected':''}>${s[0].toUpperCase()+s.slice(1)}</option>`).join('')}</select><select id="task-sort" aria-label="Sort tasks"><option value="catalog" ${filters.sort==='catalog'?'selected':''}>Catalog order</option><option value="tokens" ${filters.sort==='tokens'?'selected':''}>Most fresh input tokens</option><option value="cost" ${filters.sort==='cost'?'selected':''}>Highest cost</option><option value="name" ${filters.sort==='name'?'selected':''}>Task name</option></select></div><div class="table-scroll"><table class="task-table ${variant?'single-variant':''}"><thead><tr><th>Task</th>${variant?'<th>Native result</th><th>Fresh input<small>Excludes read and write</small></th><th>Cache read</th><th>Cache write</th><th>Cache hit</th><th>Output</th><th>Total cost<small>Estimated USD</small></th><th>Action steps</th>':data.variants.map(v=>`<th class="${v.id===filters.variant?'focus-column':''}">${v.label||v.agent}<small>${v.model} · ${v.effort}</small><small>Native result · Estimated USD</small></th>`).join('')}</tr></thead><tbody id="task-rows"></tbody></table></div><div class="table-footer"><span id="task-count" aria-live="polite"></span><span id="filter-context"></span><button class="text-button" id="reset-filters">Reset filters</button></div></div></section>`;

data-source.json

@@ -1,23 +1,23 @@
 {
   "schema": "programvla/dataset-source/1",
   "dataset": "RLE-Bench/programmable-vla",
-  "revision": "0a35b0e10a79afeaab65912a123b5448559fb7cd",
-  "evidence_root": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/0a35b0e10a79afeaab65912a123b5448559fb7cd/",
+  "revision": "c833b470484d05a4afda77fbb6bb8cf24057d4d9",
+  "evidence_root": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/c833b470484d05a4afda77fbb6bb8cf24057d4d9/",
   "files": {
     "data.json": {
-      "url": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/0a35b0e10a79afeaab65912a123b5448559fb7cd/research/oft-all140k-20261103/panels/easy/data.json"
+      "url": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/c833b470484d05a4afda77fbb6bb8cf24057d4d9/research/oft-all140k-20261103/panels/easy/data.json"
     },
     "cost-data.json": {
-      "url": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/0a35b0e10a79afeaab65912a123b5448559fb7cd/research/oft-all140k-20261103/panels/easy/cost-data.json"
+      "url": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/c833b470484d05a4afda77fbb6bb8cf24057d4d9/research/oft-all140k-20261103/panels/easy/cost-data.json"
     },
     "measured-instructions.json": {
-      "url": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/0a35b0e10a79afeaab65912a123b5448559fb7cd/research/oft-all140k-20261103/panels/easy/measured-instructions.json"
+      "url": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/c833b470484d05a4afda77fbb6bb8cf24057d4d9/research/oft-all140k-20261103/panels/easy/measured-instructions.json"
     },
     "sources.json": {
-      "url": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/0a35b0e10a79afeaab65912a123b5448559fb7cd/research/oft-all140k-20261103/panels/easy/sources.json"
+      "url": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/c833b470484d05a4afda77fbb6bb8cf24057d4d9/research/oft-all140k-20261103/panels/easy/sources.json"
     },
     "instruction-audit.json": {
-      "url": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/0a35b0e10a79afeaab65912a123b5448559fb7cd/research/oft-all140k-20261103/panels/easy/instruction-audit.json"
+      "url": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/c833b470484d05a4afda77fbb6bb8cf24057d4d9/research/oft-all140k-20261103/panels/easy/instruction-audit.json"
     }
   }
 }

hard/app.js

@@ -119,7 +119,7 @@ function opusPilotProgress(){
  return `<section class="panel" style="padding:18px 24px;margin-bottom:24px" aria-label="Claude Code campaign"><strong>Claude Code · Opus 5.5 / high</strong><p>${number(n.finished||0)} / ${number(c.planned_episodes)} episodes completed · ${number(n.running||0)} running · ${number(n.queued||0)} pending${n.needs_review?' · '+number(n.needs_review)+' under review':''}${n.interrupted?' · '+number(n.interrupted)+' interrupted':''}.</p><p class="source-note">${c.quota_waiting?'Waiting for model-service quota recovery · ':''}${number(c.published_episodes)} episode records published · <a href="${esc(c.status_url)}" target="_blank" rel="noopener">Results and protocol ↗</a></p></section>`;
 }
 function methodology() {
- return `<details class="panel methodology" id="methodology"><summary><span><span class="eyebrow">Reading the Hard benchmark</span><strong>Protocol, metrics & provenance</strong></span><span class="summary-plus">+</span></summary><div class="method-grid"><div><h3>50 tasks × 5 conditions</h3><p>RoboTwin 2.0 Hard, ALOHA AgileX, demo_randomized. Each condition is scheduled for one valid episode per task; completion is shown separately. Successes and valid failures are both retained; no valid outcome was retried. This is RoboTwin 2.0 Hard, not RoboTwin 2.0-Plus.</p></div><div><h3>Native unseen instructions</h3><p>Every episode uses the original upstream unseen instruction with its scene bindings. No RLE task03 instruction revision is applied. The exact instruction is available on each episode page and in the recorded model inputs.</p></div><div><h3>Checkpoint identity</h3><p>Base 40k: StarVLA/Qwen3-VL-OFT-Robotwin2. VLAct Clean 50k: StarVLA/VLAct_Qwen3OFT_Robotwin_Finetune. VLAct All 100k: StarVLA/VLAct_Qwen3OFT_Robotwin_all_Finetune. Non-VLAct All 140k: StarVLA/Qwen3-VL-OFT-RoboTwin2-All, trained on 27,500 clean and randomized demonstrations. Full revisions and weight checksums are linked below. Kinex uses GPT-6 Astra / high without a VLA extension.</p></div><div><h3>Execution and seeds</h3><p>Standalone VLA uses official expert admission from scene seed 100000 and official take_action with full predicted chunks. Pure Kinex uses rosbridge at 25 Hz and the first initializable seed in ascending order from 0, selected before the first model query. Seeds, admission rules and physical durations are not matched across the interfaces; this is not a paired causal comparison. Per-episode seeds and budgets are retained.</p></div><div><h3>Initialization provenance</h3><p>Five infrastructure startup attempts produced no completed model response and are excluded from the original 200 valid outcomes. For hanging_mug, a minimal runtime repair retains native language bindings if the expert path fails before finishing. Selected seed 0, task physics and the native verifier are preserved.</p></div><div><h3>Usage and costs</h3><p>${usageDefinition()} Estimated API-equivalent costs use recorded Pi session values. Actual account charges and GPU costs are not measured. Standalone VLA has zero language-agent API calls, not zero compute cost.</p></div></div><div class="method-links"><a href="${esc(evidenceURL('research/oft-all140k-20261103/protocol.json'))}">All 140k protocol & checkpoint ↗</a><a href="${esc(resultURL('sources.json'))}" target="_blank">Data sources ↗</a><a href="${esc(evidenceURL('research/robotwin-hard-20261101/REPORT.md'))}">Report ↗</a><a href="${esc(evidenceURL('research/robotwin-hard-20261101/protocol.json'))}">Protocol & checkpoints ↗</a><a href="${esc(evidenceURL('research/robotwin-hard-20261101/execution-audit.json'))}">200-episode audit ↗</a><a href="../index.html#overview">Retained Easy results ↗</a><button class="text-button" data-export="all">Download comparison CSV ↓</button></div></details>`;
+ return `<details class="panel methodology" id="methodology"><summary><span><span class="eyebrow">Reading the Hard benchmark</span><strong>Protocol, metrics & provenance</strong></span><span class="summary-plus">+</span></summary><div class="method-grid"><div><h3>50 tasks × 5 conditions</h3><p>RoboTwin 2.0 Hard, ALOHA AgileX, demo_randomized. Each condition is scheduled for one valid episode per task; completion is shown separately. Successes and valid failures are both retained; no valid outcome was retried. This is RoboTwin 2.0 Hard, not RoboTwin 2.0-Plus.</p></div><div><h3>Native unseen instructions</h3><p>Every episode uses the original upstream unseen instruction with its scene bindings. No RLE task03 instruction revision is applied. The exact instruction is available on each episode page and in the recorded model inputs.</p></div><div><h3>Checkpoint identity</h3><p>Base 40k: StarVLA/Qwen3-VL-OFT-Robotwin2. VLAct Clean 50k: StarVLA/VLAct_Qwen3OFT_Robotwin_Finetune. VLAct All 100k: StarVLA/VLAct_Qwen3OFT_Robotwin_all_Finetune. Non-VLAct All 140k: StarVLA/Qwen3-VL-OFT-RoboTwin2-All, trained on 27,500 clean and randomized demonstrations. Full revisions and weight checksums are linked below. Kinex uses GPT-6 Astra / high without a VLA extension.</p></div><div><h3>Execution and seeds</h3><p>Standalone VLA uses official expert admission from scene seed 100000 and official take_action with full predicted chunks. Pure Kinex uses rosbridge at 25 Hz and the first initializable seed in ascending order from 0, selected before the first model query. Seeds, admission rules and physical durations are not matched across the interfaces; this is not a paired causal comparison. Per-episode seeds and budgets are retained.</p></div><div><h3>Initialization provenance</h3><p>Five infrastructure startup attempts produced no completed model response and are excluded from the original 200 valid outcomes. For hanging_mug, a minimal runtime repair retains native language bindings if the expert path fails before finishing. Selected seed 0, task physics and the native verifier are preserved.</p></div><div><h3>Usage and costs</h3><p>${usageDefinition()} Estimated API-equivalent costs use recorded Pi session values. Actual account charges and GPU costs are not measured. Standalone VLA has zero language-agent API calls, not zero compute cost.</p></div></div><div class="method-links"><a href="${esc(evidenceURL('research/oft-all140k-20261103/REPORT.md'))}">All 140k results & validation ↗</a><a href="${esc(evidenceURL('research/oft-all140k-20261103/protocol.json'))}">All 140k protocol & checkpoint ↗</a><a href="${esc(resultURL('sources.json'))}" target="_blank">Data sources ↗</a><a href="${esc(evidenceURL('research/robotwin-hard-20261101/REPORT.md'))}">Report ↗</a><a href="${esc(evidenceURL('research/robotwin-hard-20261101/protocol.json'))}">Protocol & checkpoints ↗</a><a href="${esc(evidenceURL('research/robotwin-hard-20261101/execution-audit.json'))}">200-episode audit ↗</a><a href="../index.html#overview">Retained Easy results ↗</a><button class="text-button" data-export="all">Download comparison CSV ↓</button></div></details>`;
 }
 function taskBrowser(variant=null) {
   return `<section class="task-browser" id="tasks"><div class="section-heading bare"><div><div class="eyebrow">${variant?'Episode index':'Browse the benchmark'}</div><h2>Explore tasks <span class="heading-count">${plannedTasks(variant).length}</span></h2></div><button class="button secondary" data-export="${variant||'all'}">Export CSV ↓</button></div><div class="panel"><div class="filters"><label class="search-field"><span aria-hidden="true">⌕</span><input id="task-search" type="search" aria-label="Search tasks" placeholder="Search task name, instruction or ID" value="${esc(filters.query)}"></label><select id="task-family" aria-label="Environment"><option value="">All environments</option>${data.families.map(f=>`<option value="${f.id}" ${filters.family===f.id?'selected':''}>${f.name}</option>`).join('')}</select>${variant?'':`<select id="task-variant" aria-label="Agent for outcome filter">${data.variants.map(v=>`<option value="${v.id}" ${filters.variant===v.id?'selected':''}>${v.label||v.agent} · ${v.model}</option>`).join('')}</select>`}<select id="task-status" aria-label="Outcome"><option value="">All outcomes</option>${['pass','fail','pending'].map(s=>`<option value="${s}" ${filters.status===s?'selected':''}>${s[0].toUpperCase()+s.slice(1)}</option>`).join('')}</select><select id="task-sort" aria-label="Sort tasks"><option value="catalog" ${filters.sort==='catalog'?'selected':''}>Catalog order</option><option value="tokens" ${filters.sort==='tokens'?'selected':''}>Most fresh input tokens</option><option value="cost" ${filters.sort==='cost'?'selected':''}>Highest cost</option><option value="name" ${filters.sort==='name'?'selected':''}>Task name</option></select></div><div class="table-scroll"><table class="task-table ${variant?'single-variant':''}"><thead><tr><th>Task</th>${variant?'<th>Native result</th><th>Fresh input<small>Excludes read and write</small></th><th>Cache read</th><th>Cache write</th><th>Cache hit</th><th>Output</th><th>Total cost<small>Estimated USD</small></th><th>Action steps</th>':data.variants.map(v=>`<th class="${v.id===filters.variant?'focus-column':''}">${v.label||v.agent}<small>${v.model} · ${v.effort}</small><small>Native result · Estimated USD</small></th>`).join('')}</tr></thead><tbody id="task-rows"></tbody></table></div><div class="table-footer"><span id="task-count" aria-live="polite"></span><span id="filter-context"></span><button class="text-button" id="reset-filters">Reset filters</button></div></div></section>`;

hard/data-source.json

@@ -1,23 +1,23 @@
 {
   "schema": "programvla/dataset-source/1",
   "dataset": "RLE-Bench/programmable-vla",
-  "revision": "0a35b0e10a79afeaab65912a123b5448559fb7cd",
-  "evidence_root": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/0a35b0e10a79afeaab65912a123b5448559fb7cd/",
+  "revision": "c833b470484d05a4afda77fbb6bb8cf24057d4d9",
+  "evidence_root": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/c833b470484d05a4afda77fbb6bb8cf24057d4d9/",
   "files": {
     "data.json": {
-      "url": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/0a35b0e10a79afeaab65912a123b5448559fb7cd/research/oft-all140k-20261103/panels/hard/data.json"
+      "url": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/c833b470484d05a4afda77fbb6bb8cf24057d4d9/research/oft-all140k-20261103/panels/hard/data.json"
     },
     "cost-data.json": {
-      "url": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/0a35b0e10a79afeaab65912a123b5448559fb7cd/research/oft-all140k-20261103/panels/hard/cost-data.json"
+      "url": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/c833b470484d05a4afda77fbb6bb8cf24057d4d9/research/oft-all140k-20261103/panels/hard/cost-data.json"
     },
     "measured-instructions.json": {
-      "url": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/0a35b0e10a79afeaab65912a123b5448559fb7cd/research/oft-all140k-20261103/panels/hard/measured-instructions.json"
+      "url": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/c833b470484d05a4afda77fbb6bb8cf24057d4d9/research/oft-all140k-20261103/panels/hard/measured-instructions.json"
     },
     "sources.json": {
-      "url": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/0a35b0e10a79afeaab65912a123b5448559fb7cd/research/oft-all140k-20261103/panels/hard/sources.json"
+      "url": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/c833b470484d05a4afda77fbb6bb8cf24057d4d9/research/oft-all140k-20261103/panels/hard/sources.json"
     },
     "instruction-audit.json": {
-      "url": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/0a35b0e10a79afeaab65912a123b5448559fb7cd/research/oft-all140k-20261103/panels/hard/instruction-audit.json"
+      "url": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/c833b470484d05a4afda77fbb6bb8cf24057d4d9/research/oft-all140k-20261103/panels/hard/instruction-audit.json"
     }
   }
 }

hard/index.html

@@ -16,9 +16,9 @@
 <a class="skip" href="#main">Skip to content</a>
 <header class="site-header"><div class="header-inner"><a class="brand" href="#overview" aria-label="Programmable VLA home"><img src="assets/kinex-logo.svg" width="34" height="34" alt="Kinex"><span>programmable<span class="brand-divider">/</span><span class="brand-benchmark">vla</span></span></a><nav aria-label="Main navigation"><a href="../plus/index.html">Plus</a><a href="../index.html#overview">Easy</a><a href="#overview" data-nav="overview">Hard</a><a href="#tasks" data-nav="tasks">Explore tasks</a><a href="#methodology" data-nav="methodology">Protocol</a></nav><a class="header-link" href="https://huggingface.co/spaces/RLE-Bench/programmable-vla" target="_blank" rel="noopener">Hugging Face <span aria-hidden="true">↗</span></a></div></header>
 <main id="main" tabindex="-1"><div class="loading"><span class="spinner"></span>Loading benchmark results…</div></main>
-<footer class="site-footer"><div><img src="assets/kinex-logo.svg" width="22" height="22" alt=""><strong>Programmable VLA</strong><span class="footer-org">by RLE-Bench</span></div><div><a data-result-file="data.json" href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/0a35b0e10a79afeaab65912a123b5448559fb7cd/research/oft-all140k-20261103/panels/hard/data.json" target="_blank">Hard results JSON</a><a data-result-file="sources.json" href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/b07d88579523eeeefdf7ad2ca14ee65340af5d4e/hard/sources.json" target="_blank">Data sources</a><a href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/b07d88579523eeeefdf7ad2ca14ee65340af5d4e/manifest.json" target="_blank">Manifest</a><span id="updated"></span></div></footer>
+<footer class="site-footer"><div><img src="assets/kinex-logo.svg" width="22" height="22" alt=""><strong>Programmable VLA</strong><span class="footer-org">by RLE-Bench</span></div><div><a data-result-file="data.json" href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/c833b470484d05a4afda77fbb6bb8cf24057d4d9/research/oft-all140k-20261103/panels/hard/data.json" target="_blank">Hard results JSON</a><a data-result-file="sources.json" href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/b07d88579523eeeefdf7ad2ca14ee65340af5d4e/hard/sources.json" target="_blank">Data sources</a><a href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/b07d88579523eeeefdf7ad2ca14ee65340af5d4e/manifest.json" target="_blank">Manifest</a><span id="updated"></span></div></footer>
 <dialog id="image-viewer" aria-labelledby="image-caption"><div class="viewer-toolbar"><span id="image-caption"></span><a id="image-original" target="_blank" rel="noopener">Open original ↗</a><button class="icon-button" id="close-viewer" aria-label="Close image">×</button></div><div class="viewer-image"><button id="previous-image" class="viewer-arrow" aria-label="Previous image">←</button><img id="large-image" alt="Recorded agent observation"><button id="next-image" class="viewer-arrow" aria-label="Next image">→</button></div><p>Recorded agent observation · Use ← → to browse · Esc to close</p></dialog>
 <div id="toast" role="status" aria-live="polite"></div>
-<noscript><p>This site needs JavaScript. <a data-result-file="data.json" href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/0a35b0e10a79afeaab65912a123b5448559fb7cd/research/oft-all140k-20261103/panels/hard/data.json">Download the complete results index.</a></p></noscript>
+<noscript><p>This site needs JavaScript. <a data-result-file="data.json" href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/c833b470484d05a4afda77fbb6bb8cf24057d4d9/research/oft-all140k-20261103/panels/hard/data.json">Download the complete results index.</a></p></noscript>
 </body>
 </html>

index.html

@@ -16,9 +16,9 @@
 <a class="skip" href="#main">Skip to content</a>
 <header class="site-header"><div class="header-inner"><a class="brand" href="#overview" aria-label="Programmable VLA home"><img src="assets/kinex-logo.svg" width="34" height="34" alt="Kinex"><span>programmable<span class="brand-divider">/</span><span class="brand-benchmark">vla</span></span></a><nav aria-label="Main navigation"><a href="plus/index.html">Plus</a><a href="#overview" data-nav="overview">Easy</a><a href="hard/index.html#overview">Hard</a><a href="#tasks" data-nav="tasks">Explore tasks</a><a href="#methodology" data-nav="methodology">Protocol</a></nav><a class="header-link" href="https://huggingface.co/spaces/RLE-Bench/programmable-vla" target="_blank" rel="noopener">Hugging Face <span aria-hidden="true">↗</span></a></div></header>
 <main id="main" tabindex="-1"><div class="loading"><span class="spinner"></span>Loading benchmark results…</div></main>
-<footer class="site-footer"><div><img src="assets/kinex-logo.svg" width="22" height="22" alt=""><strong>Programmable VLA</strong><span class="footer-org">by RLE-Bench</span></div><div><a data-result-file="data.json" href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/0a35b0e10a79afeaab65912a123b5448559fb7cd/research/oft-all140k-20261103/panels/easy/data.json" target="_blank">Results JSON</a><a data-result-file="sources.json" href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/c2e39c9ea8f3a9d3e7680e4818d294770f032c21/sources.json" target="_blank">Data sources</a><a href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/c2e39c9ea8f3a9d3e7680e4818d294770f032c21/manifest.json" target="_blank">Manifest</a><span id="updated"></span></div></footer>
+<footer class="site-footer"><div><img src="assets/kinex-logo.svg" width="22" height="22" alt=""><strong>Programmable VLA</strong><span class="footer-org">by RLE-Bench</span></div><div><a data-result-file="data.json" href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/c833b470484d05a4afda77fbb6bb8cf24057d4d9/research/oft-all140k-20261103/panels/easy/data.json" target="_blank">Results JSON</a><a data-result-file="sources.json" href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/c2e39c9ea8f3a9d3e7680e4818d294770f032c21/sources.json" target="_blank">Data sources</a><a href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/c2e39c9ea8f3a9d3e7680e4818d294770f032c21/manifest.json" target="_blank">Manifest</a><span id="updated"></span></div></footer>
 <dialog id="image-viewer" aria-labelledby="image-caption"><div class="viewer-toolbar"><span id="image-caption"></span><a id="image-original" target="_blank" rel="noopener">Open original ↗</a><button class="icon-button" id="close-viewer" aria-label="Close image">×</button></div><div class="viewer-image"><button id="previous-image" class="viewer-arrow" aria-label="Previous image">←</button><img id="large-image" alt="Recorded agent observation"><button id="next-image" class="viewer-arrow" aria-label="Next image">→</button></div><p>Recorded agent observation · Use ← → to browse · Esc to close</p></dialog>
 <div id="toast" role="status" aria-live="polite"></div>
-<noscript><p>This site needs JavaScript. <a data-result-file="data.json" href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/0a35b0e10a79afeaab65912a123b5448559fb7cd/research/oft-all140k-20261103/panels/easy/data.json">Download the complete results index.</a></p></noscript>
+<noscript><p>This site needs JavaScript. <a data-result-file="data.json" href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/c833b470484d05a4afda77fbb6bb8cf24057d4d9/research/oft-all140k-20261103/panels/easy/data.json">Download the complete results index.</a></p></noscript>
 </body>
 </html>

manifest.json

@@ -7,8 +7,8 @@
   },
   "files": {
     "app.js": {
-      "bytes": 95483,
-      "sha256": "d79d8acbf6d2eae38c678a75c5eeaa2a0360ab86e33d79faf40ea7baa40c1431"
+      "bytes": 95593,
+      "sha256": "9891e252e9a50c2c39ff45a6ce3210ae71b9b53ba1a9b2945208284d07bd479a"
     },
     "history-app.js": {
       "bytes": 87082,
@@ -20,7 +20,7 @@
     },
     "README.md": {
       "bytes": 6808,
-      "sha256": "c77d969dd4e0eb13697d5bdc54d8b2196c95738b3351614321141353ea421488"
+      "sha256": "7fef496d700cf8318b833ce3933d9fc999a291a49e426a588341f44580de99c5"
     },
     "icon.svg": {
       "bytes": 725,
@@ -36,7 +36,7 @@
     },
     "index.html": {
       "bytes": 3413,
-      "sha256": "7791f12740fa38c4813147a60c1819a4fc6b7924a629a00cb25f02b3f4dad24c"
+      "sha256": "19dd435a6bea08e867bf7ef12e73d7256af0e99783b4a7073ba3e988e117cfbb"
     },
     "policy.css": {
       "bytes": 1067,
@@ -48,7 +48,7 @@
     },
     "data-source.json": {
       "bytes": 1376,
-      "sha256": "23f5c952280238e133f4185f7eb817e3121aa89245bb45c9b59bc364ad376821"
+      "sha256": "f2092c0812c1a8e8e56304a965e11d9ea84269902ea820e572b9f3684c5068c7"
     },
     "history/data-source.json": {
       "bytes": 2109,
@@ -75,8 +75,8 @@
       "sha256": "a53108594a23b740fbb08623f6fffdda1d73608b552988a9ffda9f0944ae66a1"
     },
     "hard/app.js": {
-      "bytes": 94287,
-      "sha256": "ac180b959b203754bd09cd7de265323dedbe00b2d9df7cf03c22a70b7d525adc"
+      "bytes": 94397,
+      "sha256": "28bce0d8868352b82e13a88708d6d890542c7ddb7d9e971ccaee78d7d8579ee8"
     },
     "hard/icon.svg": {
       "bytes": 725,
@@ -88,7 +88,7 @@
     },
     "hard/index.html": {
       "bytes": 3432,
-      "sha256": "dffd268ee07b7c68ef5cc452412d8f156e1c29ceee2a98c9b58c92de5509ee5f"
+      "sha256": "c32ba79935811093b8b6eabe6f2c92428bbcd67498f0b18c5d3a3c2834d3fe67"
     },
     "hard/policy.css": {
       "bytes": 1067,
@@ -100,7 +100,7 @@
     },
     "hard/data-source.json": {
       "bytes": 1376,
-      "sha256": "ef53075c5a17f21819ba813aae7dc693c55bd239974f4e3ab3434268b8801b79"
+      "sha256": "9322efa105ecfcda284e28336934939bdc20d6c0855a6ad327da7965b09ebd70"
     },
     "probes/bridge-20261028/index.html": {
       "bytes": 6573,
@@ -127,12 +127,12 @@
       "sha256": "a53108594a23b740fbb08623f6fffdda1d73608b552988a9ffda9f0944ae66a1"
     },
     "plus/app.js": {
-      "bytes": 95994,
-      "sha256": "ecd94e83dc8db54163b999750275af1408cfdfd0424a857c6496d2fbe5d43f8f"
+      "bytes": 96104,
+      "sha256": "2e5dd084989a265fce084e40d349c04cf805eb64fd4d6f93dd08e6d40b56ee46"
     },
     "plus/index.html": {
       "bytes": 3441,
-      "sha256": "0fc635e673fcd8caf46e99d74e1860c4bd208bcdc5dfd1e208a586aca56031d4"
+      "sha256": "2992afa00b631a27fec85fbe58be3f8974f53cef7bb658e6d7d45bccb5e7d833"
     },
     "plus/style.css": {
       "bytes": 60419,
@@ -140,7 +140,7 @@
     },
     "plus/data-source.json": {
       "bytes": 1376,
-      "sha256": "4fd0b955246e2cd5bd225cef8f4b76d250b020cac363aad30ad037ac90ae3c6c"
+      "sha256": "505ca1e94322abc111fcd4505215b1b4868f0916a192559ae7ab6f18a1b798d9"
     },
     "plus/icon.svg": {
       "bytes": 725,
@@ -189,7 +189,7 @@
   },
   "oft_all140k_dataset": {
     "repo_id": "RLE-Bench/programmable-vla",
-    "revision": "0a35b0e10a79afeaab65912a123b5448559fb7cd",
+    "revision": "c833b470484d05a4afda77fbb6bb8cf24057d4d9",
     "prefix": "research/oft-all140k-20261103"
   }
 }

plus/app.js

@@ -119,7 +119,7 @@ function opusPilotProgress(){
  return `<section class="panel" style="padding:18px 24px;margin-bottom:24px" aria-label="Claude Code campaign"><strong>Claude Code · Opus 5.5 / high</strong><p>${number(n.finished||0)} / ${number(c.planned_episodes)} episodes completed · ${number(n.running||0)} running · ${number(n.queued||0)} pending${n.needs_review?' · '+number(n.needs_review)+' under review':''}${n.interrupted?' · '+number(n.interrupted)+' interrupted':''}.</p><p class="source-note">${c.quota_waiting?'Waiting for model-service quota recovery · ':''}${number(c.published_episodes)} episode records published · <a href="${esc(c.status_url)}" target="_blank" rel="noopener">Results and protocol ↗</a></p></section>`;
 }
 function methodology() {
- return `<details class="panel methodology" id="methodology"><summary><span><span class="eyebrow">Reading the Plus benchmark</span><strong>Protocol, metrics & provenance</strong></span><span class="summary-plus">+</span></summary><div class="method-grid"><div><h3>50 tasks × 8 configurations × 4 checkpoints</h3><p>The original 1,200 valid episodes are retained. All 140k adds 400 scheduled episodes; completion is shown separately. Clean and seven perturbation configurations each contain 50 tasks per checkpoint. Overall scores include Clean. Native successes and failures are retained; no valid outcome was retried. Kinex was not scheduled for this panel.</p></div><div><h3>Native instructions</h3><p>Original Plus instructions are sent unchanged. Non-language configurations use the native seen pool; language-plus uses the native combined pool. No RLE instruction revision is applied. Exact instructions and model inputs are available on every episode page; sampled prompts may differ across checkpoints.</p></div><div><h3>Native execution and coverage</h3><p>Standalone policies use the upstream expert-success admission rule, starting at scene seed 100000 in seed group 0. Each task/configuration uses accepted episode index 0, with its actual scene seed recorded. Native take_action(qpos) executes complete predicted chunks: 16 actions for Base, 32 for VLAct, and 50 for non-VLAct All 140k. This single-episode evaluation is not the upstream multi-trial protocol. Vision-noise index 0 covers motion noise, not all noise subtypes.</p></div><div><h3>Recorded recovery</h3><p>VLAct All 100k stopped once before inference because a validator rejected angle-bracket text in a native instruction. The fix preserves whole native Plus instructions in all eight configurations while keeping custom RLE templates strict. The 116 completed All outcomes were preserved; the remaining 284 used the recorded recovery source. The interrupted attempt made zero model calls. Per-episode source phases and the recovery receipt are retained.</p></div><div><h3>Same video presentation as Easy/Hard</h3><p>Head, left wrist and right wrist appear side by side at 2880 × 720. Native 320 × 240 views are upscaled with the same H.264 CRF 20 encoding. Playback is 4× the official 10 FPS action-index recording, not physical simulation time. Wrist views hold the recorded inference observation until the next inference. Original frames and videos remain available; no simulator rerun or new camera frame was generated.</p></div><div><h3>Usage and evidence</h3><p>${usageDefinition()} These standalone runs have zero language-agent API calls; local VLA compute and GPU cost are not measured. The VLA input/output tab shows every recorded request, exact input PNG, raw response and native action chunk. Original evidence ZIPs, native videos, checkpoint revisions and source hashes are retained.</p></div></div><div class="method-links"><a href="${esc(evidenceURL('research/oft-all140k-20261103/protocol.json'))}">All 140k protocol & checkpoint ↗</a><a href="${esc(resultURL('sources.json'))}" target="_blank">Data sources ↗</a><a href="${esc(evidenceURL('robotwin-plus/20261101-native/REPORT.md'))}">Report ↗</a><a href="${esc(evidenceURL('robotwin-plus/20261101-native/protocol.json'))}">Protocol & checkpoints ↗</a><a href="${esc(evidenceURL('robotwin-plus/20261101-native/recovery/receipt.json'))}">Recovery receipt ↗</a><a href="${esc(evidenceURL('robotwin-plus/20261101-native/presentation-v2/presentation-protocol.json'))}">Video presentation protocol ↗</a><button class="text-button" data-export="all">Download comparison CSV ↓</button></div></details>`;
+ return `<details class="panel methodology" id="methodology"><summary><span><span class="eyebrow">Reading the Plus benchmark</span><strong>Protocol, metrics & provenance</strong></span><span class="summary-plus">+</span></summary><div class="method-grid"><div><h3>50 tasks × 8 configurations × 4 checkpoints</h3><p>The original 1,200 valid episodes are retained. All 140k adds 400 scheduled episodes; completion is shown separately. Clean and seven perturbation configurations each contain 50 tasks per checkpoint. Overall scores include Clean. Native successes and failures are retained; no valid outcome was retried. Kinex was not scheduled for this panel.</p></div><div><h3>Native instructions</h3><p>Original Plus instructions are sent unchanged. Non-language configurations use the native seen pool; language-plus uses the native combined pool. No RLE instruction revision is applied. Exact instructions and model inputs are available on every episode page; sampled prompts may differ across checkpoints.</p></div><div><h3>Native execution and coverage</h3><p>Standalone policies use the upstream expert-success admission rule, starting at scene seed 100000 in seed group 0. Each task/configuration uses accepted episode index 0, with its actual scene seed recorded. Native take_action(qpos) executes complete predicted chunks: 16 actions for Base, 32 for VLAct, and 50 for non-VLAct All 140k. This single-episode evaluation is not the upstream multi-trial protocol. Vision-noise index 0 covers motion noise, not all noise subtypes.</p></div><div><h3>Recorded recovery</h3><p>VLAct All 100k stopped once before inference because a validator rejected angle-bracket text in a native instruction. The fix preserves whole native Plus instructions in all eight configurations while keeping custom RLE templates strict. The 116 completed All outcomes were preserved; the remaining 284 used the recorded recovery source. The interrupted attempt made zero model calls. Per-episode source phases and the recovery receipt are retained.</p></div><div><h3>Same video presentation as Easy/Hard</h3><p>Head, left wrist and right wrist appear side by side at 2880 × 720. Native 320 × 240 views are upscaled with the same H.264 CRF 20 encoding. Playback is 4× the official 10 FPS action-index recording, not physical simulation time. Wrist views hold the recorded inference observation until the next inference. Original frames and videos remain available; no simulator rerun or new camera frame was generated.</p></div><div><h3>Usage and evidence</h3><p>${usageDefinition()} These standalone runs have zero language-agent API calls; local VLA compute and GPU cost are not measured. The VLA input/output tab shows every recorded request, exact input PNG, raw response and native action chunk. Original evidence ZIPs, native videos, checkpoint revisions and source hashes are retained.</p></div></div><div class="method-links"><a href="${esc(evidenceURL('research/oft-all140k-20261103/REPORT.md'))}">All 140k results & validation ↗</a><a href="${esc(evidenceURL('research/oft-all140k-20261103/protocol.json'))}">All 140k protocol & checkpoint ↗</a><a href="${esc(resultURL('sources.json'))}" target="_blank">Data sources ↗</a><a href="${esc(evidenceURL('robotwin-plus/20261101-native/REPORT.md'))}">Report ↗</a><a href="${esc(evidenceURL('robotwin-plus/20261101-native/protocol.json'))}">Protocol & checkpoints ↗</a><a href="${esc(evidenceURL('robotwin-plus/20261101-native/recovery/receipt.json'))}">Recovery receipt ↗</a><a href="${esc(evidenceURL('robotwin-plus/20261101-native/presentation-v2/presentation-protocol.json'))}">Video presentation protocol ↗</a><button class="text-button" data-export="all">Download comparison CSV ↓</button></div></details>`;
 }
 function taskBrowser(variant=null) {
   return `<section class="task-browser" id="tasks"><div class="section-heading bare"><div><div class="eyebrow">${variant?'Episode index':'Browse the benchmark'}</div><h2>Explore tasks <span class="heading-count">${plannedTasks(variant).length}</span></h2></div><button class="button secondary" data-export="${variant||'all'}">Export CSV ↓</button></div><div class="panel"><div class="filters"><label class="search-field"><span aria-hidden="true">⌕</span><input id="task-search" type="search" aria-label="Search tasks" placeholder="Search task name, instruction or ID" value="${esc(filters.query)}"></label><select id="task-family" aria-label="Configuration"><option value="">All configurations</option>${data.families.map(f=>`<option value="${f.id}" ${filters.family===f.id?'selected':''}>${f.name}</option>`).join('')}</select>${variant?'':`<select id="task-variant" aria-label="Agent for outcome filter">${data.variants.map(v=>`<option value="${v.id}" ${filters.variant===v.id?'selected':''}>${v.label||v.agent} · ${v.model}</option>`).join('')}</select>`}<select id="task-status" aria-label="Outcome"><option value="">All outcomes</option>${['pass','fail','pending'].map(s=>`<option value="${s}" ${filters.status===s?'selected':''}>${s[0].toUpperCase()+s.slice(1)}</option>`).join('')}</select><select id="task-sort" aria-label="Sort tasks"><option value="catalog" ${filters.sort==='catalog'?'selected':''}>Catalog order</option><option value="tokens" ${filters.sort==='tokens'?'selected':''}>Most fresh input tokens</option><option value="cost" ${filters.sort==='cost'?'selected':''}>Highest cost</option><option value="name" ${filters.sort==='name'?'selected':''}>Task name</option></select></div><div class="table-scroll"><table class="task-table ${variant?'single-variant':''}"><thead><tr><th>Task</th>${variant?'<th>Native result</th><th>Fresh input<small>Excludes read and write</small></th><th>Cache read</th><th>Cache write</th><th>Cache hit</th><th>Output</th><th>Total cost<small>Estimated USD</small></th><th>Action steps</th>':data.variants.map(v=>`<th class="${v.id===filters.variant?'focus-column':''}">${v.label||v.agent}<small>${v.model} · ${v.effort}</small><small>Native result · Estimated USD</small></th>`).join('')}</tr></thead><tbody id="task-rows"></tbody></table></div><div class="table-footer"><span id="task-count" aria-live="polite"></span><span id="filter-context"></span><button class="text-button" id="reset-filters">Reset filters</button></div></div></section>`;

plus/data-source.json

@@ -1,23 +1,23 @@
 {
   "schema": "programvla/dataset-source/1",
   "dataset": "RLE-Bench/programmable-vla",
-  "revision": "0a35b0e10a79afeaab65912a123b5448559fb7cd",
-  "evidence_root": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/0a35b0e10a79afeaab65912a123b5448559fb7cd/",
+  "revision": "c833b470484d05a4afda77fbb6bb8cf24057d4d9",
+  "evidence_root": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/c833b470484d05a4afda77fbb6bb8cf24057d4d9/",
   "files": {
     "data.json": {
-      "url": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/0a35b0e10a79afeaab65912a123b5448559fb7cd/research/oft-all140k-20261103/panels/plus/data.json"
+      "url": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/c833b470484d05a4afda77fbb6bb8cf24057d4d9/research/oft-all140k-20261103/panels/plus/data.json"
     },
     "cost-data.json": {
-      "url": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/0a35b0e10a79afeaab65912a123b5448559fb7cd/research/oft-all140k-20261103/panels/plus/cost-data.json"
+      "url": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/c833b470484d05a4afda77fbb6bb8cf24057d4d9/research/oft-all140k-20261103/panels/plus/cost-data.json"
     },
     "measured-instructions.json": {
-      "url": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/0a35b0e10a79afeaab65912a123b5448559fb7cd/research/oft-all140k-20261103/panels/plus/measured-instructions.json"
+      "url": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/c833b470484d05a4afda77fbb6bb8cf24057d4d9/research/oft-all140k-20261103/panels/plus/measured-instructions.json"
     },
     "sources.json": {
-      "url": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/0a35b0e10a79afeaab65912a123b5448559fb7cd/research/oft-all140k-20261103/panels/plus/sources.json"
+      "url": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/c833b470484d05a4afda77fbb6bb8cf24057d4d9/research/oft-all140k-20261103/panels/plus/sources.json"
     },
     "instruction-audit.json": {
-      "url": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/0a35b0e10a79afeaab65912a123b5448559fb7cd/research/oft-all140k-20261103/panels/plus/instruction-audit.json"
+      "url": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/c833b470484d05a4afda77fbb6bb8cf24057d4d9/research/oft-all140k-20261103/panels/plus/instruction-audit.json"
     }
   }
 }

plus/index.html

@@ -16,9 +16,9 @@
 <a class="skip" href="#main">Skip to content</a>
 <header class="site-header"><div class="header-inner"><a class="brand" href="#overview" aria-label="Programmable VLA home"><img src="assets/kinex-logo.svg" width="34" height="34" alt="Kinex"><span>programmable<span class="brand-divider">/</span><span class="brand-benchmark">vla</span></span></a><nav aria-label="Main navigation"><a href="../index.html#overview">Easy</a><a href="../hard/index.html#overview">Hard</a><a href="#overview" data-nav="overview">Plus</a><a href="#tasks" data-nav="tasks">Explore tasks</a><a href="#methodology" data-nav="methodology">Protocol</a></nav><a class="header-link" href="https://huggingface.co/spaces/RLE-Bench/programmable-vla" target="_blank" rel="noopener">Hugging Face <span aria-hidden="true">↗</span></a></div></header>
 <main id="main" tabindex="-1"><div class="loading"><span class="spinner"></span>Loading benchmark results…</div></main>
-<footer class="site-footer"><div><img src="assets/kinex-logo.svg" width="22" height="22" alt=""><strong>Programmable VLA</strong><span class="footer-org">by RLE-Bench</span></div><div><a data-result-file="data.json" href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/0a35b0e10a79afeaab65912a123b5448559fb7cd/research/oft-all140k-20261103/panels/plus/data.json" target="_blank">Plus results JSON</a><a data-result-file="sources.json" href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/b07d88579523eeeefdf7ad2ca14ee65340af5d4e/hard/sources.json" target="_blank">Data sources</a><a href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/b2d67a67dd2f9d8e12c1f8db13ba8de309739883/manifest.json" target="_blank">Manifest</a><span id="updated"></span></div></footer>
+<footer class="site-footer"><div><img src="assets/kinex-logo.svg" width="22" height="22" alt=""><strong>Programmable VLA</strong><span class="footer-org">by RLE-Bench</span></div><div><a data-result-file="data.json" href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/c833b470484d05a4afda77fbb6bb8cf24057d4d9/research/oft-all140k-20261103/panels/plus/data.json" target="_blank">Plus results JSON</a><a data-result-file="sources.json" href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/b07d88579523eeeefdf7ad2ca14ee65340af5d4e/hard/sources.json" target="_blank">Data sources</a><a href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/b2d67a67dd2f9d8e12c1f8db13ba8de309739883/manifest.json" target="_blank">Manifest</a><span id="updated"></span></div></footer>
 <dialog id="image-viewer" aria-labelledby="image-caption"><div class="viewer-toolbar"><span id="image-caption"></span><a id="image-original" target="_blank" rel="noopener">Open original ↗</a><button class="icon-button" id="close-viewer" aria-label="Close image">×</button></div><div class="viewer-image"><button id="previous-image" class="viewer-arrow" aria-label="Previous image">←</button><img id="large-image" alt="Recorded agent observation"><button id="next-image" class="viewer-arrow" aria-label="Next image">→</button></div><p>Recorded agent observation · Use ← → to browse · Esc to close</p></dialog>
 <div id="toast" role="status" aria-live="polite"></div>
-<noscript><p>This site needs JavaScript. <a data-result-file="data.json" href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/0a35b0e10a79afeaab65912a123b5448559fb7cd/research/oft-all140k-20261103/panels/plus/data.json">Download the complete results index.</a></p></noscript>
+<noscript><p>This site needs JavaScript. <a data-result-file="data.json" href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/c833b470484d05a4afda77fbb6bb8cf24057d4d9/research/oft-all140k-20261103/panels/plus/data.json">Download the complete results index.</a></p></noscript>
 </body>
 </html>