Spaces:
Running
Running
Publish RoboTwin 2.0-Plus results: 1200 native VLA episodes
Browse files- README.md +6 -0
- hard/index.html +1 -1
- index.html +1 -1
- manifest.json +27 -6
- plus/app.js +8 -0
- plus/data-source.json +6 -0
- plus/index.html +9 -0
- plus/style.css +1 -0
README.md
CHANGED
|
@@ -40,3 +40,9 @@ This lightweight Space contains presentation code, logos and revision pointers o
|
|
| 40 |
## RoboTwin 2.0 Hard · November 1, 2026
|
| 41 |
|
| 42 |
[Open the Hard results](https://rle-bench-programmable-vla.static.hf.space/hard/index.html): 50 tasks per condition, one valid episode each, native unseen instructions. Base 40k **2/50**, VLAct Clean **16/50**, VLAct All **46/50**, pure Kinex Astra/high **33/50**. The existing Easy and historical panels are retained. Execution interfaces, seeds and physical budgets differ; the panel documents those limits.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 40 |
## RoboTwin 2.0 Hard · November 1, 2026
|
| 41 |
|
| 42 |
[Open the Hard results](https://rle-bench-programmable-vla.static.hf.space/hard/index.html): 50 tasks per condition, one valid episode each, native unseen instructions. Base 40k **2/50**, VLAct Clean **16/50**, VLAct All **46/50**, pure Kinex Astra/high **33/50**. The existing Easy and historical panels are retained. Execution interfaces, seeds and physical budgets differ; the panel documents those limits.
|
| 43 |
+
|
| 44 |
+
## RoboTwin 2.0-Plus · November 2, 2026
|
| 45 |
+
|
| 46 |
+
[Open the Plus results](https://rle-bench-programmable-vla.static.hf.space/plus/index.html): **1,200 valid episodes**, three checkpoints × 50 tasks × eight configurations × one admitted episode, with native instructions. Base 40k **94/400 (23.5%)**, VLAct Clean 50k **222/400 (55.5%)**, VLAct All 100k **294/400 (73.5%)**. All episodes have native videos and full evidence ZIPs. Existing Easy, Hard and historical pages are retained.
|
| 47 |
+
|
| 48 |
+
Single-episode coverage is not the upstream multi-trial protocol; vision-noise index 0 covers motion noise only. The page and [report](https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/9f8a68a40148e1ad013b35a2d8774ff113cb0c5e/robotwin-plus/20261101-native/REPORT.md) document expert admission, native language pools, checkpoint identities and the All recovery. No valid outcomes were retried. Dataset revision: `9f8a68a40148e1ad013b35a2d8774ff113cb0c5e`.
|
hard/index.html
CHANGED
|
@@ -14,7 +14,7 @@
|
|
| 14 |
</head>
|
| 15 |
<body>
|
| 16 |
<a class="skip" href="#main">Skip to content</a>
|
| 17 |
-
<header class="site-header"><div class="header-inner"><a class="brand" href="#overview" aria-label="Programmable VLA home"><img src="assets/kinex-logo.svg" width="34" height="34" alt="Kinex"><span>programmable<span class="brand-divider">/</span><span class="brand-benchmark">vla</span></span></a><nav aria-label="Main navigation"><a href="../index.html#overview">Easy</a><a href="#overview" data-nav="overview">Hard</a><a href="#tasks" data-nav="tasks">Explore tasks</a><a href="#methodology" data-nav="methodology">Protocol</a></nav><a class="header-link" href="https://huggingface.co/spaces/RLE-Bench/programmable-vla" target="_blank" rel="noopener">Hugging Face <span aria-hidden="true">↗</span></a></div></header>
|
| 18 |
<main id="main" tabindex="-1"><div class="loading"><span class="spinner"></span>Loading benchmark results…</div></main>
|
| 19 |
<footer class="site-footer"><div><img src="assets/kinex-logo.svg" width="22" height="22" alt=""><strong>Programmable VLA</strong><span class="footer-org">by RLE-Bench</span></div><div><a data-result-file="data.json" href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/b07d88579523eeeefdf7ad2ca14ee65340af5d4e/hard/data.json" target="_blank">Hard results JSON</a><a data-result-file="sources.json" href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/b07d88579523eeeefdf7ad2ca14ee65340af5d4e/hard/sources.json" target="_blank">Data sources</a><a href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/b07d88579523eeeefdf7ad2ca14ee65340af5d4e/manifest.json" target="_blank">Manifest</a><span id="updated"></span></div></footer>
|
| 20 |
<dialog id="image-viewer" aria-labelledby="image-caption"><div class="viewer-toolbar"><span id="image-caption"></span><a id="image-original" target="_blank" rel="noopener">Open original ↗</a><button class="icon-button" id="close-viewer" aria-label="Close image">×</button></div><div class="viewer-image"><button id="previous-image" class="viewer-arrow" aria-label="Previous image">←</button><img id="large-image" alt="Recorded agent observation"><button id="next-image" class="viewer-arrow" aria-label="Next image">→</button></div><p>Recorded agent observation · Use ← → to browse · Esc to close</p></dialog>
|
|
|
|
| 14 |
</head>
|
| 15 |
<body>
|
| 16 |
<a class="skip" href="#main">Skip to content</a>
|
| 17 |
+
<header class="site-header"><div class="header-inner"><a class="brand" href="#overview" aria-label="Programmable VLA home"><img src="assets/kinex-logo.svg" width="34" height="34" alt="Kinex"><span>programmable<span class="brand-divider">/</span><span class="brand-benchmark">vla</span></span></a><nav aria-label="Main navigation"><a href="../plus/index.html">Plus</a><a href="../index.html#overview">Easy</a><a href="#overview" data-nav="overview">Hard</a><a href="#tasks" data-nav="tasks">Explore tasks</a><a href="#methodology" data-nav="methodology">Protocol</a></nav><a class="header-link" href="https://huggingface.co/spaces/RLE-Bench/programmable-vla" target="_blank" rel="noopener">Hugging Face <span aria-hidden="true">↗</span></a></div></header>
|
| 18 |
<main id="main" tabindex="-1"><div class="loading"><span class="spinner"></span>Loading benchmark results…</div></main>
|
| 19 |
<footer class="site-footer"><div><img src="assets/kinex-logo.svg" width="22" height="22" alt=""><strong>Programmable VLA</strong><span class="footer-org">by RLE-Bench</span></div><div><a data-result-file="data.json" href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/b07d88579523eeeefdf7ad2ca14ee65340af5d4e/hard/data.json" target="_blank">Hard results JSON</a><a data-result-file="sources.json" href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/b07d88579523eeeefdf7ad2ca14ee65340af5d4e/hard/sources.json" target="_blank">Data sources</a><a href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/b07d88579523eeeefdf7ad2ca14ee65340af5d4e/manifest.json" target="_blank">Manifest</a><span id="updated"></span></div></footer>
|
| 20 |
<dialog id="image-viewer" aria-labelledby="image-caption"><div class="viewer-toolbar"><span id="image-caption"></span><a id="image-original" target="_blank" rel="noopener">Open original ↗</a><button class="icon-button" id="close-viewer" aria-label="Close image">×</button></div><div class="viewer-image"><button id="previous-image" class="viewer-arrow" aria-label="Previous image">←</button><img id="large-image" alt="Recorded agent observation"><button id="next-image" class="viewer-arrow" aria-label="Next image">→</button></div><p>Recorded agent observation · Use ← → to browse · Esc to close</p></dialog>
|
index.html
CHANGED
|
@@ -14,7 +14,7 @@
|
|
| 14 |
</head>
|
| 15 |
<body>
|
| 16 |
<a class="skip" href="#main">Skip to content</a>
|
| 17 |
-
<header class="site-header"><div class="header-inner"><a class="brand" href="#overview" aria-label="Programmable VLA home"><img src="assets/kinex-logo.svg" width="34" height="34" alt="Kinex"><span>programmable<span class="brand-divider">/</span><span class="brand-benchmark">vla</span></span></a><nav aria-label="Main navigation"><a href="#overview" data-nav="overview">Easy</a><a href="hard/index.html#overview">Hard</a><a href="#tasks" data-nav="tasks">Explore tasks</a><a href="#methodology" data-nav="methodology">Protocol</a></nav><a class="header-link" href="https://huggingface.co/spaces/RLE-Bench/programmable-vla" target="_blank" rel="noopener">Hugging Face <span aria-hidden="true">↗</span></a></div></header>
|
| 18 |
<main id="main" tabindex="-1"><div class="loading"><span class="spinner"></span>Loading benchmark results…</div></main>
|
| 19 |
<footer class="site-footer"><div><img src="assets/kinex-logo.svg" width="22" height="22" alt=""><strong>Programmable VLA</strong><span class="footer-org">by RLE-Bench</span></div><div><a data-result-file="data.json" href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/c2e39c9ea8f3a9d3e7680e4818d294770f032c21/data.json" target="_blank">Results JSON</a><a data-result-file="sources.json" href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/c2e39c9ea8f3a9d3e7680e4818d294770f032c21/sources.json" target="_blank">Data sources</a><a href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/c2e39c9ea8f3a9d3e7680e4818d294770f032c21/manifest.json" target="_blank">Manifest</a><span id="updated"></span></div></footer>
|
| 20 |
<dialog id="image-viewer" aria-labelledby="image-caption"><div class="viewer-toolbar"><span id="image-caption"></span><a id="image-original" target="_blank" rel="noopener">Open original ↗</a><button class="icon-button" id="close-viewer" aria-label="Close image">×</button></div><div class="viewer-image"><button id="previous-image" class="viewer-arrow" aria-label="Previous image">←</button><img id="large-image" alt="Recorded agent observation"><button id="next-image" class="viewer-arrow" aria-label="Next image">→</button></div><p>Recorded agent observation · Use ← → to browse · Esc to close</p></dialog>
|
|
|
|
| 14 |
</head>
|
| 15 |
<body>
|
| 16 |
<a class="skip" href="#main">Skip to content</a>
|
| 17 |
+
<header class="site-header"><div class="header-inner"><a class="brand" href="#overview" aria-label="Programmable VLA home"><img src="assets/kinex-logo.svg" width="34" height="34" alt="Kinex"><span>programmable<span class="brand-divider">/</span><span class="brand-benchmark">vla</span></span></a><nav aria-label="Main navigation"><a href="plus/index.html">Plus</a><a href="#overview" data-nav="overview">Easy</a><a href="hard/index.html#overview">Hard</a><a href="#tasks" data-nav="tasks">Explore tasks</a><a href="#methodology" data-nav="methodology">Protocol</a></nav><a class="header-link" href="https://huggingface.co/spaces/RLE-Bench/programmable-vla" target="_blank" rel="noopener">Hugging Face <span aria-hidden="true">↗</span></a></div></header>
|
| 18 |
<main id="main" tabindex="-1"><div class="loading"><span class="spinner"></span>Loading benchmark results…</div></main>
|
| 19 |
<footer class="site-footer"><div><img src="assets/kinex-logo.svg" width="22" height="22" alt=""><strong>Programmable VLA</strong><span class="footer-org">by RLE-Bench</span></div><div><a data-result-file="data.json" href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/c2e39c9ea8f3a9d3e7680e4818d294770f032c21/data.json" target="_blank">Results JSON</a><a data-result-file="sources.json" href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/c2e39c9ea8f3a9d3e7680e4818d294770f032c21/sources.json" target="_blank">Data sources</a><a href="https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/c2e39c9ea8f3a9d3e7680e4818d294770f032c21/manifest.json" target="_blank">Manifest</a><span id="updated"></span></div></footer>
|
| 20 |
<dialog id="image-viewer" aria-labelledby="image-caption"><div class="viewer-toolbar"><span id="image-caption"></span><a id="image-original" target="_blank" rel="noopener">Open original ↗</a><button class="icon-button" id="close-viewer" aria-label="Close image">×</button></div><div class="viewer-image"><button id="previous-image" class="viewer-arrow" aria-label="Previous image">←</button><img id="large-image" alt="Recorded agent observation"><button id="next-image" class="viewer-arrow" aria-label="Next image">→</button></div><p>Recorded agent observation · Use ← → to browse · Esc to close</p></dialog>
|
manifest.json
CHANGED
|
@@ -19,8 +19,8 @@
|
|
| 19 |
"sha256": "34a4d6e437537e78bc242418805a3af3d71b43980280482fd8fc6bfda91f79f8"
|
| 20 |
},
|
| 21 |
"README.md": {
|
| 22 |
-
"bytes":
|
| 23 |
-
"sha256": "
|
| 24 |
},
|
| 25 |
"icon.svg": {
|
| 26 |
"bytes": 725,
|
|
@@ -35,8 +35,8 @@
|
|
| 35 |
"sha256": "8b0691838063224bce2a991d7bb0672897ef8f51bbc39ce38afb4a67cb6e39d0"
|
| 36 |
},
|
| 37 |
"index.html": {
|
| 38 |
-
"bytes":
|
| 39 |
-
"sha256": "
|
| 40 |
},
|
| 41 |
"policy.css": {
|
| 42 |
"bytes": 1067,
|
|
@@ -87,8 +87,8 @@
|
|
| 87 |
"sha256": "330e92f8eb9670cb506f2909a2a47ee8f0bd569199898df3bfa044239ae64161"
|
| 88 |
},
|
| 89 |
"hard/index.html": {
|
| 90 |
-
"bytes":
|
| 91 |
-
"sha256": "
|
| 92 |
},
|
| 93 |
"hard/policy.css": {
|
| 94 |
"bytes": 1067,
|
|
@@ -125,6 +125,22 @@
|
|
| 125 |
"hard/assets/anthropic-logo.svg": {
|
| 126 |
"bytes": 352,
|
| 127 |
"sha256": "a53108594a23b740fbb08623f6fffdda1d73608b552988a9ffda9f0944ae66a1"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 128 |
}
|
| 129 |
},
|
| 130 |
"hard_dataset": {
|
|
@@ -132,5 +148,10 @@
|
|
| 132 |
"revision": "b07d88579523eeeefdf7ad2ca14ee65340af5d4e",
|
| 133 |
"manifest": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/b07d88579523eeeefdf7ad2ca14ee65340af5d4e/manifest.json",
|
| 134 |
"results": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/b07d88579523eeeefdf7ad2ca14ee65340af5d4e/hard/data.json"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 135 |
}
|
| 136 |
}
|
|
|
|
| 19 |
"sha256": "34a4d6e437537e78bc242418805a3af3d71b43980280482fd8fc6bfda91f79f8"
|
| 20 |
},
|
| 21 |
"README.md": {
|
| 22 |
+
"bytes": 5223,
|
| 23 |
+
"sha256": "dc6236f40830ce1cd37734c33e46562b6f5ac5864fd58c2675c85dfae0ce701f"
|
| 24 |
},
|
| 25 |
"icon.svg": {
|
| 26 |
"bytes": 725,
|
|
|
|
| 35 |
"sha256": "8b0691838063224bce2a991d7bb0672897ef8f51bbc39ce38afb4a67cb6e39d0"
|
| 36 |
},
|
| 37 |
"index.html": {
|
| 38 |
+
"bytes": 3333,
|
| 39 |
+
"sha256": "5c60da97019262ace5b0335f3d41c7c6ca9a8f7b6cff16abe5745df1213ba13f"
|
| 40 |
},
|
| 41 |
"policy.css": {
|
| 42 |
"bytes": 1067,
|
|
|
|
| 87 |
"sha256": "330e92f8eb9670cb506f2909a2a47ee8f0bd569199898df3bfa044239ae64161"
|
| 88 |
},
|
| 89 |
"hard/index.html": {
|
| 90 |
+
"bytes": 3351,
|
| 91 |
+
"sha256": "5ab0faa63b0141aad0ea623006bd93f4ca688262a2ea84852e968cf27b2876c5"
|
| 92 |
},
|
| 93 |
"hard/policy.css": {
|
| 94 |
"bytes": 1067,
|
|
|
|
| 125 |
"hard/assets/anthropic-logo.svg": {
|
| 126 |
"bytes": 352,
|
| 127 |
"sha256": "a53108594a23b740fbb08623f6fffdda1d73608b552988a9ffda9f0944ae66a1"
|
| 128 |
+
},
|
| 129 |
+
"plus/app.js": {
|
| 130 |
+
"bytes": 4928,
|
| 131 |
+
"sha256": "845cacc1806f0c46e13b1386a5921679342b19bc9d96bda5fdd8ee8a7b239edd"
|
| 132 |
+
},
|
| 133 |
+
"plus/index.html": {
|
| 134 |
+
"bytes": 4480,
|
| 135 |
+
"sha256": "3b58ffb145850d1cc2e06868f9470f50a51a5101e1f2971738c795d3f6de7f8b"
|
| 136 |
+
},
|
| 137 |
+
"plus/style.css": {
|
| 138 |
+
"bytes": 4263,
|
| 139 |
+
"sha256": "0c0ba971b534d0d07bf6a60d42b0d4c11c6e19a08f8f54aa6a5923c5f2743a81"
|
| 140 |
+
},
|
| 141 |
+
"plus/data-source.json": {
|
| 142 |
+
"bytes": 279,
|
| 143 |
+
"sha256": "344981106e0120c9b743556145977ce4b533012cba8d644ad60fe8e85cecfc8c"
|
| 144 |
}
|
| 145 |
},
|
| 146 |
"hard_dataset": {
|
|
|
|
| 148 |
"revision": "b07d88579523eeeefdf7ad2ca14ee65340af5d4e",
|
| 149 |
"manifest": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/b07d88579523eeeefdf7ad2ca14ee65340af5d4e/manifest.json",
|
| 150 |
"results": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/b07d88579523eeeefdf7ad2ca14ee65340af5d4e/hard/data.json"
|
| 151 |
+
},
|
| 152 |
+
"plus_dataset": {
|
| 153 |
+
"repo_id": "RLE-Bench/programmable-vla",
|
| 154 |
+
"revision": "9f8a68a40148e1ad013b35a2d8774ff113cb0c5e",
|
| 155 |
+
"prefix": "robotwin-plus/20261101-native"
|
| 156 |
}
|
| 157 |
}
|
plus/app.js
ADDED
|
@@ -0,0 +1,8 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"use strict";
|
| 2 |
+
const $=s=>document.querySelector(s), names={"base40k":"Base 40k","vlact-clean50k":"VLAct Clean 50k","vlact-all100k":"VLAct All 100k"}, configs={demo_clean:"Clean",demo_background_plus:"Background+",demo_camera:"Camera",demo_language_plus:"Language+",demo_light:"Light",demo_objects_plus:"Objects+",demo_robot_state:"Robot state",demo_vision_noise:"Vision noise"};
|
| 3 |
+
const esc=s=>String(s).replace(/[&<>"']/g,c=>({"&":"&","<":"<",">":">",'"':""","'":"'"}[c]));
|
| 4 |
+
let data,root,selected;
|
| 5 |
+
function detail(e){selected=e.id;const metrics=[["Native scene seed",e.seed],["Accepted episode index",e.now_ep_num],["Native control steps",`${e.native_steps} / ${e.native_step_limit}`],["Policy calls",e.inference_calls],["Episode wall time",`${e.wall_time_s.toFixed(1)} s`],["Simulation time",`${e.simulation_time_s.toFixed(1)} s`]];$("#detail").innerHTML=`<h3>${esc(e.task_name.replaceAll("_"," "))}</h3><div><span class="verdict ${e.success?"success":"failure"}">${e.success?"Success":"Failure"}</span> · ${esc(names[e.variant])} · ${esc(configs[e.config])}</div><video controls preload="metadata" playsinline src="${root+e.video}"></video><small>Native recorded rollout · playback follows the source video timeline.</small><h4>Exact native instruction sent to the policy</h4><p class="instruction">${esc(e.instruction)}</p><div class="metrics">${metrics.map(([k,v])=>`<div><span>${esc(k)}</span>${esc(v)}</div>`).join("")}</div><p><small>Source phase: ${e.source_phase==="recovery"?"recorded recovery source":"original frozen source"}. Timing starts after native setup and ends at the last action.</small></p><a class="download" href="${root+e.evidence}" target="_blank" rel="noopener">Download full evidence ZIP ↗</a><a href="${root+e.video}" target="_blank" rel="noopener">Open video ↗</a>`;for(const row of document.querySelectorAll("#tasks tr"))row.classList.toggle("selected",row.dataset.id===e.id);history.replaceState(null,"",`#episode=${encodeURIComponent(e.id)}`);}
|
| 6 |
+
function filter(){const rows=data.episodes.filter(e=>e.variant===$("#model").value&&e.config===$("#config").value&&e.task_name.replaceAll("_"," ").includes($("#search").value.toLowerCase().replaceAll("_"," ")));$("#count").textContent=`${rows.length} episodes shown · ${rows.filter(e=>e.success).length} native successes`;$("#tasks").innerHTML=rows.map(e=>`<tr data-id="${esc(e.id)}"><td><button type="button">${esc(e.task_name.replaceAll("_"," "))}</button></td><td class="${e.success?"success":"failure"}">${e.success?"Success":"Failure"}</td></tr>`).join("");[...document.querySelectorAll("#tasks tr")].forEach((tr,i)=>tr.querySelector("button").onclick=()=>detail(rows[i]));if(rows.length)detail(rows.find(e=>e.id===selected)||rows[0]);else $("#detail").innerHTML="<p>No matching task.</p>";}
|
| 7 |
+
async function init(){const ptr=await fetch("data-source.json").then(r=>{if(!r.ok)throw Error("Missing Dataset pointer");return r.json()});root=ptr.root;data=await fetch(root+ptr.data).then(r=>{if(!r.ok)throw Error("Cannot load results");return r.json()});window.plusData=data;$("#cards").innerHTML=Object.entries(names).map(([v,n])=>{const x=data.conditions[v];return `<article class="card"><h2>${esc(n)}</h2><div class="score">${(100*x.success/x.valid).toFixed(1)}%</div><p>${x.success} / ${x.valid} successes</p><small>${x.valid} / ${x.planned} valid episodes</small></article>`}).join("");$("#scores").innerHTML=Object.entries(configs).map(([c,n])=>`<tr><td>${n}</td>${Object.keys(names).map(v=>{const r=data.branches.find(x=>x.variant===v&&x.config===c);return `<td>${r.success}/${r.valid}<small>${(100*r.success/r.valid).toFixed(0)}%</small></td>`}).join("")}</tr>`).join("");$("#model").innerHTML=Object.entries(names).map(([v,n])=>`<option value="${v}">${n}</option>`).join("");$("#config").innerHTML=Object.entries(configs).map(([v,n])=>`<option value="${v}">${n}</option>`).join("");$("#model").value="vlact-all100k";const id=location.hash.startsWith("#episode=")?decodeURIComponent(location.hash.slice(9)):null;const chosen=data.episodes.find(e=>e.id===id);if(chosen){selected=chosen.id;$("#model").value=chosen.variant;$("#config").value=chosen.config;}for(const s of ["#model","#config"])$(s).onchange=filter;$("#search").oninput=filter;for(const [id,path] of Object.entries({report:"REPORT.md",json:"data.json","protocol-file":"protocol.json",sources:"source-identities.json",recovery:"recovery/receipt.json"}))$("#"+id).href=root+data.campaign+"/"+path;$("#checkpoints").innerHTML=Object.entries(data.protocol.checkpoints).map(([v,c])=>`<p><strong>${names[v]}</strong> · <a href="https://huggingface.co/${c.repo}/tree/${c.revision}" target="_blank" rel="noopener">${c.repo} ↗</a><br>Checkpoint revision: <code>${c.revision}</code></p>`).join("");filter();}
|
| 8 |
+
init().catch(e=>{$("#load-error").textContent=`Results could not load: ${e.message}. Please refresh or open the evidence Dataset.`;console.error(e);});
|
plus/data-source.json
ADDED
|
@@ -0,0 +1,6 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"repo_id": "RLE-Bench/programmable-vla",
|
| 3 |
+
"revision": "9f8a68a40148e1ad013b35a2d8774ff113cb0c5e",
|
| 4 |
+
"root": "https://huggingface.co/datasets/RLE-Bench/programmable-vla/resolve/9f8a68a40148e1ad013b35a2d8774ff113cb0c5e/",
|
| 5 |
+
"data": "robotwin-plus/20261101-native/data.json"
|
| 6 |
+
}
|
plus/index.html
ADDED
|
@@ -0,0 +1,9 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
<!doctype html>
|
| 2 |
+
<html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"><meta name="theme-color" content="#756ee4"><title>RoboTwin 2.0-Plus · Programmable VLA</title><link rel="icon" href="../icon.svg"><link rel="stylesheet" href="style.css"><script defer src="app.js"></script></head><body>
|
| 3 |
+
<header><a class="brand" href="../index.html"><img src="../assets/kinex-logo.svg" alt="" width="30" height="30"> programmable<span>/</span>vla</a><nav aria-label="Main navigation"><a href="../index.html">Easy</a><a href="../hard/index.html">Hard</a><a href="index.html" aria-current="page">Plus</a><a href="#protocol">Protocol</a></nav></header>
|
| 4 |
+
<main><div class="eyebrow">RLE-BENCH / COMPLETED · NOVEMBER 2, 2026</div><h1>RoboTwin 2.0-Plus</h1><p class="lead">Three VLA checkpoints. 50 tasks × 8 configurations.<br>One admitted episode per task and configuration, with native instructions.</p>
|
| 5 |
+
<section id="cards" class="cards" aria-label="Overall results"></section>
|
| 6 |
+
<section class="panel"><h2>Results by configuration</h2><p>Each cell is native successes / 50 valid episodes. The overall score includes Clean.</p><div class="table-scroll"><table><thead><tr><th>Configuration</th><th>Base 40k</th><th>VLAct Clean 50k</th><th>VLAct All 100k</th></tr></thead><tbody id="scores"></tbody></table></div></section>
|
| 7 |
+
<section id="explore" class="panel"><h2>Explore the 1,200 episodes</h2><div class="filters"><label>Checkpoint<select id="model"></select></label><label>Configuration<select id="config"></select></label><label>Task name<input id="search" type="search" placeholder="Search tasks"></label></div><p id="count" aria-live="polite"></p><div class="explorer"><div class="task-list"><table><thead><tr><th>Task</th><th>Native verdict</th></tr></thead><tbody id="tasks"></tbody></table></div><article id="detail" aria-live="polite"><p>Loading episode…</p></article></div></section>
|
| 8 |
+
<section id="protocol" class="panel"><h2>Protocol & evidence</h2><p>Standalone frozen-policy evaluation through the native RoboTwin evaluator. Native <code>take_action(qpos)</code> executes complete predicted action chunks: 16 for Base and 32 for VLAct. Native task limits and native success verdicts are retained.</p><p>Seed group 0 starts at scene seed 100000; the upstream expert-success admission rule selects the accepted scene in ascending seed order. Each task/configuration uses accepted episode index 0. Actual seeds appear in episode details. Native instructions are passed unchanged; prompts may differ across models. Non-language configurations use the native seen pool, while language-plus uses the native combined pool.</p><p><strong>This is a single-episode coverage evaluation, not the upstream multi-trial paper protocol.</strong> Vision noise at index 0 covers motion noise only, not all noise subtypes. No Kinex evaluation is included in this panel.</p><details><summary>Recovery provenance · no valid outcomes retried</summary><p>VLAct All had one infrastructure interruption before model inference: a validator treated angle-bracket text in a native instruction as an unresolved placeholder. The fix preserves whole native Plus instructions in all eight configurations while keeping custom RLE templates strict. The first 116 completed All results were preserved; the remaining 284 used the recorded recovery source. The interrupted attempt made zero model calls. No valid success or failure was retried.</p><a id="recovery" target="_blank" rel="noopener">Recovery receipt ↗</a></details><p>Each evidence ZIP includes the native verdict, setup/admission events, exact instruction, configuration, all policy input images, model responses, native actions and video. File manifests distinguish original frozen hashes from public hashes after host-path redaction.</p><p class="links"><a id="report" target="_blank" rel="noopener">Full report ↗</a><a id="json" target="_blank" rel="noopener">Results JSON ↗</a><a id="protocol-file" target="_blank" rel="noopener">Protocol JSON ↗</a><a id="sources" target="_blank" rel="noopener">Source identities ↗</a></p><div id="checkpoints"></div></section>
|
| 9 |
+
<p id="load-error" role="alert"></p></main><footer><a href="https://huggingface.co/spaces/RLE-Bench/programmable-vla">Hugging Face Space ↗</a><a href="https://huggingface.co/datasets/RLE-Bench/programmable-vla">Evidence Dataset ↗</a><span>Existing Easy, Hard and historical panels retained.</span></footer></body></html>
|
plus/style.css
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
*{box-sizing:border-box}html{scroll-behavior:smooth}body{margin:0;background:#f8f9fc;color:#222437;font:15px/1.65 system-ui,-apple-system,sans-serif}a{color:#5c52c6;text-decoration:none}a:hover{text-decoration:underline}header{background:#fff;border-bottom:1px solid #e3e4ee;padding:20px max(24px,calc((100vw - 1240px)/2));display:flex;justify-content:space-between;align-items:center;gap:24px}.brand{display:flex;align-items:center;gap:8px;color:#292b40;font-weight:750;font-size:20px}.brand span{color:#9794c8}nav{display:flex;gap:25px}nav a{color:#62657a}nav [aria-current]{color:#5c52c6;font-weight:750}main{max-width:1240px;margin:56px auto;padding:0 24px}.eyebrow{font-size:12px;letter-spacing:.12em;font-weight:700;color:#70689f}h1{font-size:clamp(32px,5vw,52px);line-height:1.2;letter-spacing:-.04em;margin:14px 0}h2{font-size:22px;line-height:1.3;margin:0 0 12px}h3{font-size:19px;margin:0 0 12px}.lead{font-size:19px;color:#666b81;margin-bottom:30px}.cards{display:grid;grid-template-columns:repeat(3,minmax(0,1fr));gap:18px;margin:30px 0}.card,.panel{background:#fff;border:1px solid #e2e4ef;border-radius:14px;padding:25px}.card h2{font-size:16px}.score{font-size:38px;font-weight:750;letter-spacing:-.04em}.card p{margin:2px 0;color:#6d7184}.panel{margin:24px 0}.panel>p{color:#64697e}.table-scroll{overflow:auto}table{border-collapse:collapse;width:100%;text-align:left;font-variant-numeric:tabular-nums}th{font-size:12px;text-transform:uppercase;letter-spacing:.04em;color:#74788a;background:#f8f9fc}td,th{padding:11px 14px;border-bottom:1px solid #eef0f5}td small{color:#74788a;margin-left:8px}button{cursor:pointer;font:inherit;border:0;background:transparent;color:#544ac0;padding:0;text-align:left}button:hover{text-decoration:underline}.filters{display:flex;gap:16px;flex-wrap:wrap;margin-top:24px}label{display:flex;flex-direction:column;gap:5px;font-size:12px;color:#656a81;font-weight:600}select,input{font:14px system-ui;padding:11px 12px;border:1px solid #d7dae7;border-radius:7px;background:#fff;color:#262a40;min-width:200px}.explorer{display:grid;grid-template-columns:1fr 1.15fr;gap:26px}.task-list{max-height:720px;overflow:auto;border:1px solid #e5e7f0;border-radius:9px}.task-list th{position:sticky;top:0}.task-list tr.selected{background:#f0edff}.success{color:#218153}.failure{color:#a64b59}.verdict{font-weight:650}#detail{min-width:0;border-left:1px solid #e6e8f1;padding-left:26px}video{display:block;width:100%;max-height:350px;background:#151727;border-radius:8px;margin:16px 0}.instruction{background:#f5f3fc;border-left:3px solid #8b80db;padding:13px 16px;color:#42415b;overflow-wrap:anywhere}.metrics{display:grid;grid-template-columns:1fr 1fr;gap:8px;font-size:13px;margin:18px 0}.metrics div{background:#f8f9fc;padding:9px 12px;border-radius:6px}.metrics span{display:block;color:#777c90;font-size:11px;text-transform:uppercase}.download{display:inline-block;background:#665bd0;color:#fff;padding:10px 15px;border-radius:7px;margin:10px 12px 8px 0}small{color:#777c90}.links{display:flex;gap:20px;flex-wrap:wrap}details{background:#f8f9fc;border:1px solid #e4e6f0;border-radius:8px;padding:14px 18px;margin:18px 0}summary{cursor:pointer;font-weight:600}code{background:#f2f2f8;padding:2px 5px;border-radius:3px}footer{max-width:1240px;margin:30px auto;padding:24px;display:flex;gap:24px;flex-wrap:wrap;border-top:1px solid #dddfea;color:#777c90;font-size:13px}#checkpoints{font-size:13px;overflow-wrap:anywhere}#load-error{color:#a64b59}@media(max-width:800px){header{padding:16px;flex-wrap:wrap;gap:12px}.brand{font-size:18px}nav{gap:20px}main{margin-top:32px;padding:0 16px}.cards{gap:9px}.card{padding:14px}.card h2{font-size:13px}.score{font-size:28px}.panel{padding:18px}.explorer{grid-template-columns:1fr}.task-list{max-height:330px}#detail{border-left:0;border-top:1px solid #e6e8f1;padding:22px 0 0}td,th{padding:9px;font-size:13px}.filters{display:grid;grid-template-columns:1fr}.filters label{min-width:0}.table-scroll table{table-layout:fixed}.table-scroll th{overflow-wrap:anywhere;font-size:10px}.table-scroll th:first-child{width:31%}.table-scroll td small{display:block;margin:0}select,input{min-width:0;width:100%}.lead{font-size:16px}}@media(prefers-reduced-motion:reduce){html{scroll-behavior:auto}}
|