`;
}
function recordingPilotProgress(){
const p=data.recording_pilot;if(!p)return '';
return `Codex + RoboBridge · GPT-6 Astra / high · GPU recording pilot
${number(p.completed)} / ${number(p.planned)} independent attempts published: LIBERO, RoboTwin, and two RoboDojo recording conditions. This pilot does not cover the full benchmark.
LIBERO and RoboTwin use independent 720-high spectator cameras. RoboDojo compares native camera recording with online HD recording. Both keep native agent camera resolution, but the RTX control test found that extra HD cameras change subsequent native image pixels. The HD control is shown separately.
Videos play at 4× simulation speed. Session galleries contain images actually submitted to the agent. One model episode per condition cannot establish a success-rate effect.
${c.quota_waiting?'Waiting for model-service quota recovery · ':''}${number(c.published_episodes)} episode records published · Results and protocol ↗
`;
}
function methodology() {
return `Reading the Plus benchmarkProtocol, metrics & provenance+
50 tasks × 8 configurations × 4 checkpoints
The original 1,200 valid episodes are retained. All 140k adds 400 scheduled episodes; completion is shown separately. Clean and seven perturbation configurations each contain 50 tasks per checkpoint. Overall scores include Clean. Native successes and failures are retained; no valid outcome was retried. Pure Kinex adds 24 selected episodes: handover_block, scan_object and adjust_bottle in each of the eight configurations. The other 376 pairs are not scheduled for Kinex.
Instruction protocols
Standalone VLA retains original Plus instructions: the native seen pool for non-language configurations and the combined pool for language-plus. Pure Kinex uses revised RLE templates, including the RLE language-plus revision. Exact instructions and model inputs are available on every episode page; sampled prompts may differ across checkpoints.
Native execution and coverage
Standalone policies use the upstream expert-success admission rule, starting at scene seed 100000 in seed group 0. Each task/configuration uses accepted episode index 0, with its actual scene seed recorded. Native take_action(qpos) executes complete predicted chunks: 16 actions for Base, 32 for VLAct, and 50 for non-VLAct All 140k. This single-episode evaluation is not the upstream multi-trial protocol. Vision-noise index 0 covers motion noise, not all noise subtypes.
Recorded recovery
VLAct All 100k stopped once before inference because a validator rejected angle-bracket text in a native instruction. The fix preserves whole native Plus instructions in all eight configurations while keeping custom RLE templates strict. The 116 completed All outcomes were preserved; the remaining 284 used the recorded recovery source. The interrupted attempt made zero model calls. Per-episode source phases and the recovery receipt are retained.
Same video presentation as Easy/Hard
Head, left wrist and right wrist appear side by side at 2880 × 720. Kinex retains all three spectator recordings at 4× simulation speed. The following action-index details apply to standalone VLA only. Native 320 × 240 views are upscaled with the same H.264 CRF 20 encoding. Playback is 4× the official 10 FPS action-index recording, not physical simulation time. Wrist views hold the recorded inference observation until the next inference. Original frames and videos remain available; no simulator rerun or new camera frame was generated.
Usage and evidence
Standalone runs have zero language-agent API calls; Kinex records provider usage and API-equivalent estimates from its sessions; local VLA compute and GPU cost are not measured. The VLA input/output tab shows every recorded request, exact input PNG, raw response and native action chunk. Original evidence ZIPs, native videos, checkpoint revisions and source hashes are retained.
`;
}
const expandedTaskVariants = new Set();
const visibleTaskVariants = () => data.variants.filter(v => expandedTaskVariants.has(v.id));
function taskColumnPicker() {
return `
Choose methods to compare
All method columns are collapsed. Select one or more to compare.
${data.variants.map(v=>``).join('')}
`;
}
function taskColumnHeaders() {
return '
Task
' + visibleTaskVariants().map(v=>`
${esc(v.label||v.agent)}
${esc(v.model)} · ${esc(v.effort)}Native result · Estimated USD
`).join('');
}
function syncTaskColumns() {
if(current.type==='runs')return;
const variants=visibleTaskVariants(), table=$('#task-comparison');
table.querySelector('thead tr').innerHTML=taskColumnHeaders();
table.classList.toggle('has-methods',variants.length>0);
table.style.setProperty('--method-count',variants.length);
$$('[data-task-column].task-column-toggle').forEach(button=>{
const expanded=expandedTaskVariants.has(button.dataset.taskColumn);
button.setAttribute('aria-expanded',expanded);
button.querySelector('.column-toggle-icon').textContent=expanded?'−':'+';
});
$('#task-column-status').textContent=variants.length?`${variants.length} of ${data.variants.length} method columns expanded. Select another method to compare.`:'All method columns are collapsed. Select one or more to compare.';
$('[data-task-columns="none"]').disabled=variants.length===0;
$('[data-task-columns="all"]').disabled=variants.length===data.variants.length;
}
function taskBrowser(variant=null) {
return `
`;
$('#task-count').textContent=`${tasks.length} of ${data.tasks.length} tasks`;
$('#filter-context').textContent=variant?variantName(variant):'Outcome filter & usage/cost sort: '+variantName(filters.variant);
}
function researchRerunNote() {
const r=data.research_update;if(!r)return '';
return ``;
}
function overview() {
current={type:'overview'};
$('#main').innerHTML=`
✦ RLE-Bench · Embodied agent evaluation
RoboTwin 2.0-Plus. Plus evaluation.
Eight configurations · four VLA checkpoints retain native instructions. Pure Kinex evaluates three tasks per configuration with RLE revised instructions.
${data.families.length}configurations
${data.tasks.length}task/configuration pairs
${data.variants.length}agent–model variants
${data.benchmark_complete?'Evaluation complete':'Evaluation in progress'} · selected seeds
`;
}
// Align words and punctuation while preserving the exact original whitespace.
function instructionDiff(original, revised) {
const split = text => text.match(/\s+|[\p{L}\p{N}_]+|[^\s]/gu) || [];
const a=split(original), b=split(revised);
const lcs=Array.from({length:a.length+1},()=>new Uint16Array(b.length+1));
for(let i=a.length-1;i>=0;i--)for(let j=b.length-1;j>=0;j--)
lcs[i][j]=a[i]===b[j]?lcs[i+1][j+1]+1:Math.max(lcs[i+1][j],lcs[i][j+1]);
const before=[],after=[];
const append=(list,text,changed)=>{const last=list.at(-1);if(last&&last.changed===changed)last.text+=text;else list.push({text,changed});};
let i=0,j=0;
while(i=lcs[i+1][j]))append(after,b[j++],true);
else append(before,a[i++],true);
}
const html=(parts,kind)=>parts.map(p=>p.changed&&p.text.trim()?`<${kind==='removed'?'del':'mark'} class="instruction-${kind}">${esc(p.text)}`:esc(p.text)).join('');
return {original:html(before,'removed'),revised:html(after,'added')};
}
function instructionPair(task, ep) {
const candidate = ep ? measuredInstructions.episodes[ep.id] : null;
const record = candidate && candidate.initial_goal === ep.instruction && candidate.task_key === task.key ? candidate : null;
const recordedNative = record?.observed_native || (ep ? ep.native_instruction : task.native_instruction);
const measured = record?.text || (ep ? (ep.instruction || (ep.instruction_policy === 'original_native' ? recordedNative : null)) : null);
const reference = catalogInstructionAudit.tasks.find(item => item.key === task.key && item.native_id === task.native_id);
const nativeReference = !recordedNative && reference?.native ? reference : null;
const native = recordedNative || nativeReference?.native;
return {record, native, measured, nativeReference};
}
function instructionPanel(task,ep) {
const {record,native,measured:recorded,nativeReference}=instructionPair(task,ep);
const measured=ep?recorded:(task.catalog_instruction_by_variant?.[current.variant]||task.catalog_instruction);
const initial=record&&record.initial_goal!==measured?record.initial_goal:null;
const changed=Boolean(native&&measured&&native!==measured);
const diff=changed?instructionDiff(native,measured):null;
const choices=[['measured',ep?'Measured':'Planned',measured],['native','Original native',native],['current','Current catalog',null]].filter(([key,,text])=>text&&(key!=='current'||(ep&&text!==measured&&text!==initial)));
const currentChanged=native&&task.catalog_instruction&&native!==task.catalog_instruction;
const description=!ep?'The current catalog instruction for the planned evaluation.':record?.kind==='scene_bound_goal'?"The measured goal with this episode's objects and targets filled in from the agent's session.":record?.kind==='native_received'?'The native instruction received by the agent during this episode.':'The task instruction supplied for this episode.';
return `
Task instruction
${description}
${choices.map(([k,label],i)=>``).join('')}
${nativeReference?`
Native comparison uses a pinned upstream reference${ep?'; native text was not recorded in this episode':''}. ${external(nativeReference.source_url,'Native source')}
`:''}${choices.map(([k,,text],i)=>{
const comparison=k==='current'?(currentChanged?instructionDiff(native,text):null):diff;
const html=comparison?(k==='native'?comparison.original:comparison.revised):esc(text);
const note=comparison?(k==='native'?'Struck-through text was removed or replaced in the '+(ep?'measured':'planned')+' instruction.':'Highlighted text was added or replaced relative to the original native instruction.'):(native?'No changes from the original native instruction.':'Original native text is unavailable for comparison.');
return `
${comparison?``:''}${note}${k==='current'?' This catalog text is not the measured episode instruction.':''}
${html}
`;
}).join('')}${record&&initial?`
${record.kind==='scene_bound_goal'?'Object names are resolved for display; the measured completion requirements are preserved.':'The initial input contained a generic placeholder; this native instruction was received in the recorded session.'} Original instruction record ↗ · Session evidence ↗
Original harness verdict: invalid. Recording manifest: incomplete. This attempt is excluded from formal success metrics; its usage is retained separately.
${seedLimit?.exhausted?'No native-valid seed at or below '+number(seedLimit.max_seed):initializationBlocked?'Native seed 0 could not initialize':'No recorded episode yet'}
${seedLimit?.exhausted?'Native checks exhausted seeds 1–'+number(seedLimit.max_seed)+'. No Codex model attempt was launched. The original Kinex result is retained; this unavailable Codex task is excluded from success rates.':initializationBlocked?'No model was called. The authored seed could not initialize; a seed-selection decision is pending. This case is excluded from native success rates.':'This task is included in the planned catalog. Results and usage will appear after evaluation.'}
${m.playback_kind==='official_action_index'?'Head + left wrist + right wrist · 4× official recording, not simulation time. Wrist views update only at inference boundaries. Native 320 × 240 views are upscaled for display; action-limit recordings omit the post-final-action frame.':ep.recording_mode==='native'?'Native camera recording · cached agent-resolution frames · '+(m.speed||1)+'× simulation speed.':ep.provider==='robobridge'&&m.playback_kind==='live_spectator'?'Independent spectator cameras · 720 px high per view · overview + wrist views · '+(m.speed||1)+'× simulation speed. Agent images retain native resolution and orientation.':m.playback_kind==='native_observation_snapshots'?'Native observation snapshots · 10 FPS playback; this is not simulation time.':`${m.source_fps?m.source_fps+' FPS source capture · ':''}${m.speed||1}× simulation speed.`} Session timestamps use wall time and are not synchronized to video frames.
No standalone tool or memo files were saved in this episode. The original session and workspace evidence remain available.
'}
${evidenceCard('Execution evidence','Original verdict, protocol and runtime identity.',[['verdict','Native verdict'],['protocol','Protocol'],['provenance','Source identity']],ep)}${evidenceCard('Session & interactions','Original tool calls, observations and edited workspace.',[['native_session','Native session'],['trace','Trace'],['workspace_changes','Workspace changes']],ep)}
`;
}
async function fetchText(url) {
if(fileCache.has(url))return fileCache.get(url);
const r=await fetch(url);if(!r.ok)throw Error('HTTP '+r.status);
if(Number(r.headers.get('content-length'))>3e6)throw Error('File exceeds the 3 MB preview limit; use Open file');
const text=await r.text();if(text.length>3e6)throw Error('File exceeds the preview limit; use Open file');
fileCache.set(url,text);if(fileCache.size>20)fileCache.delete(fileCache.keys().next().value);return text;
}
async function loadResource(index,token) {
const r=current.ep?.resources?.[index];if(!r)return;
current.resource=index;$('#resource-name').textContent=r.name;$('#resource-open').href=r.file;$('#resource-kind').textContent=r.kind||'Final workspace snapshot';$('#resource-preview').textContent='Loading file…';
$$('[data-resource]').forEach(b=>{b.classList.toggle('active',+b.dataset.resource===index);b.setAttribute('aria-pressed',+b.dataset.resource===index);});
try{const text=await fetchText(r.file);if(token===epoch&¤t.resource===index){$('#resource-preview').textContent=text;current.resourceText=text;}}
catch(e){if(token===epoch&¤t.resource===index){$('#resource-preview').textContent='Preview unavailable: '+e.message;current.resourceText=null;}}
}
const evidenceGroups=[
['Native evaluation','Owner verdicts, measured task and control protocol.',[['verdict','Native verdict'],['timing','Episode timing'],['protocol','Measured protocol'],['native_goal','Task instructions'],['owner_journal','Owner action journal']]],
['Session & workspace','The original agent trajectory and final files.',[['native_session','Native session'],['transcript','Readable transcript'],['trace','Agent trace'],['workspace','Workspace archive'],['workspace_changes','Workspace manifest']]],
['Recording & review','Published media and checks against the recorded evidence.',[['native_video','Original head recording'],['video','Video'],['poster','Poster'],['terminal_frame','Terminal frame'],['recording_manifest','Recording manifest'],['media_check','Media validation'],['analysis','Episode analysis']]],
['Usage & provenance','Provider counters and the actual code used by the episode.',[['usage','Token usage'],['provider_usage','Provider usage ledger'],['provenance','Source identities'],['tool_errors','Tool errors']]]
];
function evidenceCard(title,description,items,ep) {
return `
${esc(c.call_id)}${c.service_response.inference_s.toFixed(3)} s · ${(c.service_response.waypoints||c.service_response.actions).length} × ${(c.service_response.waypoints||c.service_response.actions)[0].length} actions
Task instruction
${esc(c.request.task)}
${images}
${esc(record.contract.image_preprocessing||"Model images follow the recorded checkpoint preprocessing.")} Additional information reaches the model only when present in the request and rendered language below.
${jsonBlock('Complete model-service request',c.request)}${c.model_input_text!==undefined?jsonBlock('Language passed to predict_action · recorded',c.model_input_text,true):''}${jsonBlock('Backbone message · reconstructed from executed code',c.backbone_messages)}
Model-service output
${c.service_response.waypoints?'Untimed joint waypoints consumed by the recorded caller. Kinex or agent-authored code determines their physical timing.':'The recorded action response consumed by the caller.'} Joint values are absolute radians; grippers use −1 open, +1 closed.
${jsonBlock('Complete service response · full precision',c.service_response)}${c.execution?jsonBlock('Actual execution · target count and world time',{executor:c.planner,...c.execution},true):''}${jsonBlock('Internal diagnostic · denormalized training-order actions',c.internal_diagnostics.denormalized_actions)}${caller?`
Kinex tool call & result
This inference is joined to the recorded tool call and result below. Agent-authored code may consume the model output directly; the language agent receives the displayed tool response.
${jsonBlock('Agent tool call',caller.tool_call,true)}${jsonBlock('Tool result received by the language agent',{...caller.tool_result,parts:textParts(caller.tool_result.parts)},true)}
${c.executed_tool_source?jsonBlock('Agent-authored code at this call',c.executed_tool_source):''}`:'
Standalone execution; no language-agent tool call.
'}Recording coverage & model contract
See the per-run contract below for recorded and reconstructed fields. Original requests and rendered language are retained in the current rich-input run; older runs have different recording coverage.
${jsonBlock('Complete recorded StarVLA response · full precision',c.raw_response)}${jsonBlock('Reordered native actions · full precision',c.service_response.actions)}
Standalone execution; no language-agent session or tool call. Token and API costs cover the language agent only; local GPU cost is not reported.
${jsonBlock('Recording coverage & model contract',r.contract)}
`;
function stats(es) {
return UsageMetrics.stats(es);
}
window.benchmarkStats = stats;
const tokens = e => e && Number.isFinite(e.usage?.input_tokens) && Number.isFinite(e.usage?.output_tokens) ? e.usage.input_tokens+e.usage.output_tokens : null;
function metric(value,label,detail='',attrs='') {return `