`;
}
function recordingComparisonNote(ep){
if(!ep.recording_comparison)return '';
const hd=ep.recording_mode==='spectator';
return ``;
}
function methodology() {
return `Reading the benchmarkProtocol, metrics & provenance+
One selected episode
Each standalone checkpoint: 50 tasks, one episode each, official seed group 0 with native expert admission. VLAct Clean 50K, VLAct All 100K and the original Base 40K episodes are all retained. Kinex: the retained three-task seed-0 pilot. Kinex + VLA: the October 30 VLAct Clean 50k run on those same three tasks and seed 0; previous valid attempts remain archived.
Native success
Success rate is passes / evaluated tasks, pooled over the selected episodes. Pending tasks do not count as failures. Results follow the original native verifier.
Native control budget
RLE-Bench task03 · RoboTwin 2.0 · ALOHA AgileX. Standalone uses official take_action and task-specific limits. Kinex uses rosbridge control frames. Initializations and physical durations are not matched.
Policy & agent
Non-VLAct All 140k: StarVLA/Qwen3-VL-OFT-RoboTwin2-All, step 140,000, trained on 27,500 clean/randomized demonstrations. It uses three 224 × 224 images and full 50-action chunks. The existing Easy instruction protocol is retained; expert admission matched Base scene seeds on 45/50 tasks.
Standalone VLAct Clean: StarVLA/VLAct_Qwen3OFT_Robotwin_Finetune, step 50,000, RoboTwin Clean/Base training mixture. It uses the same released 32-action, image-bucket and wrap-joint contract as VLAct All. Official expert admission selected the same scene seed as All in 45/50 tasks. Each checkpoint has one admitted episode per task; training mixture and training duration differ, so these results do not isolate a causal mechanism.
Standalone VLAct All: StarVLA/VLAct_Qwen3OFT_Robotwin_all_Finetune, step 100,000, 32-action chunks and checkpoint-specific image buckets / wrap decoding. Official expert admission selected the same scene seed as Base 40k in 43 of 50 tasks; this is not a fully paired comparison. Standalone Base 40k retains StarVLA/Qwen3-VL-OFT-Robotwin2, step 40,000. Kinex + VLA now uses VLAct Clean 50k, with 32 predicted targets and its checkpoint-specific image buckets and wrap decoder. Kinex uses GPT-6 Astra / high. Kinex chooses whether to use VLA, writes its control loop, and chooses language, arbitrary JSON conditions and optional formatting. Additional conditions enter the frozen model through language. This interface supports richer inputs; effectiveness is a separate experimental question. The rerun returns untimed joint waypoints, executed by Kinex trajectories or agent-authored code. Encoder changes and progress monitors are not used.
Token accounting
API-equivalent cost uses the recorded Pi session estimates; account charges and GPU costs are not reported.
`;
}
const expandedTaskVariants = new Set();
const visibleTaskVariants = () => data.variants.filter(v => expandedTaskVariants.has(v.id));
function taskColumnPicker() {
return `
Choose methods to compare
All method columns are collapsed. Select one or more to compare.
${data.variants.map(v=>``).join('')}
`;
}
function taskColumnHeaders() {
return '
Task
' + visibleTaskVariants().map(v=>`
${esc(v.label||v.agent)}
${esc(v.model)} · ${esc(v.effort)}Native result · Estimated USD
`).join('');
}
function syncTaskColumns() {
if(current.type==='runs')return;
const variants=visibleTaskVariants(), table=$('#task-comparison');
table.querySelector('thead tr').innerHTML=taskColumnHeaders();
table.classList.toggle('has-methods',variants.length>0);
table.style.setProperty('--method-count',variants.length);
$$('[data-task-column].task-column-toggle').forEach(button=>{
const expanded=expandedTaskVariants.has(button.dataset.taskColumn);
button.setAttribute('aria-expanded',expanded);
button.querySelector('.column-toggle-icon').textContent=expanded?'−':'+';
});
$('#task-column-status').textContent=variants.length?`${variants.length} of ${data.variants.length} method columns expanded. Select another method to compare.`:'All method columns are collapsed. Select one or more to compare.';
$('[data-task-columns="none"]').disabled=variants.length===0;
$('[data-task-columns="all"]').disabled=variants.length===data.variants.length;
}
function taskBrowser(variant=null) {
return `
`;
}
// Align words and punctuation while preserving the exact original whitespace.
function instructionDiff(original, revised) {
const split = text => text.match(/\s+|[\p{L}\p{N}_]+|[^\s]/gu) || [];
const a=split(original), b=split(revised);
const lcs=Array.from({length:a.length+1},()=>new Uint16Array(b.length+1));
for(let i=a.length-1;i>=0;i--)for(let j=b.length-1;j>=0;j--)
lcs[i][j]=a[i]===b[j]?lcs[i+1][j+1]+1:Math.max(lcs[i+1][j],lcs[i][j+1]);
const before=[],after=[];
const append=(list,text,changed)=>{const last=list.at(-1);if(last&&last.changed===changed)last.text+=text;else list.push({text,changed});};
let i=0,j=0;
while(i=lcs[i+1][j]))append(after,b[j++],true);
else append(before,a[i++],true);
}
const html=(parts,kind)=>parts.map(p=>p.changed&&p.text.trim()?`<${kind==='removed'?'del':'mark'} class="instruction-${kind}">${esc(p.text)}`:esc(p.text)).join('');
return {original:html(before,'removed'),revised:html(after,'added')};
}
function instructionPair(task, ep) {
const candidate = ep ? measuredInstructions.episodes[ep.id] : null;
const record = candidate && candidate.initial_goal === ep.instruction && candidate.task_key === task.key ? candidate : null;
const recordedNative = record?.observed_native || (ep ? ep.native_instruction : task.native_instruction);
const measured = record?.text || (ep ? (ep.instruction || (ep.instruction_policy === 'original_native' ? recordedNative : null)) : null);
const reference = catalogInstructionAudit.tasks.find(item => item.key === task.key && item.native_id === task.native_id);
const nativeReference = !recordedNative && reference?.native ? reference : null;
const native = recordedNative || nativeReference?.native;
return {record, native, measured, nativeReference};
}
function instructionPanel(task,ep) {
const {record,native,measured:recorded,nativeReference}=instructionPair(task,ep);
const measured=ep?recorded:task.catalog_instruction;
const initial=record&&record.initial_goal!==measured?record.initial_goal:null;
const changed=Boolean(native&&measured&&native!==measured);
const diff=changed?instructionDiff(native,measured):null;
const choices=[['measured',ep?'Measured':'Planned',measured],['native','Original native',native],['current','Current catalog',task.catalog_instruction]].filter(([key,,text])=>text&&(key!=='current'||(ep&&text!==measured&&text!==initial)));
const currentChanged=native&&task.catalog_instruction&&native!==task.catalog_instruction;
const description=!ep?'The current catalog instruction for the planned evaluation.':record?.kind==='scene_bound_goal'?"The measured goal with this episode's objects and targets filled in from the agent's session.":record?.kind==='native_received'?'The native instruction received by the agent during this episode.':'The task instruction supplied for this episode.';
return `
Task instruction
${description}
${choices.map(([k,label],i)=>``).join('')}
${nativeReference?`
Native comparison uses a pinned upstream reference${ep?'; native text was not recorded in this episode':''}. ${external(nativeReference.source_url,'Native source')}
`:''}${choices.map(([k,,text],i)=>{
const comparison=k==='current'?(currentChanged?instructionDiff(native,text):null):diff;
const html=comparison?(k==='native'?comparison.original:comparison.revised):esc(text);
const note=comparison?(k==='native'?'Struck-through text was removed or replaced in the '+(ep?'measured':'planned')+' instruction.':'Highlighted text was added or replaced relative to the original native instruction.'):(native?'No changes from the original native instruction.':'Original native text is unavailable for comparison.');
return `
${comparison?``:''}${note}${k==='current'?' This catalog text is not the measured episode instruction.':''}
${html}
`;
}).join('')}${record&&initial?`
${record.kind==='scene_bound_goal'?'Object names are resolved for display; the measured completion requirements are preserved.':'The initial input contained a generic placeholder; this native instruction was received in the recorded session.'} Original instruction record ↗ · Session evidence ↗
Original harness verdict: invalid. Recording manifest: incomplete. This attempt is excluded from formal success metrics; its usage is retained separately.
${seedLimit?.exhausted?'No native-valid seed at or below '+number(seedLimit.max_seed):initializationBlocked?'Native seed 0 could not initialize':'No recorded episode yet'}
${seedLimit?.exhausted?'Native checks exhausted seeds 1–'+number(seedLimit.max_seed)+'. No Codex model attempt was launched. The original Kinex result is retained; this unavailable Codex task is excluded from success rates.':initializationBlocked?'No model was called. Seed 0 is retained by user decision; this is excluded from task success rates.':'This task is included in the planned catalog. Results and usage will appear after evaluation.'}
${m.playback_kind==='official_action_index'?'Head + left wrist + right wrist · 4× official recording, not simulation time. Wrist views update only at inference boundaries. Native 320 × 240 views are upscaled for display; action-limit recordings omit the post-final-action frame.':ep.recording_mode==='native'?'Native camera recording · cached agent-resolution frames · '+(m.speed||1)+'× simulation speed.':ep.provider==='robobridge'&&m.playback_kind==='live_spectator'?'Independent spectator cameras · 720 px high per view · overview + wrist views · '+(m.speed||1)+'× simulation speed. Agent images retain native resolution and orientation.':m.playback_kind==='native_observation_snapshots'?'Native observation snapshots · 10 FPS playback; this is not simulation time.':`${m.source_fps?m.source_fps+' FPS source capture · ':''}${m.speed||1}× simulation speed.`} Session timestamps use wall time and are not synchronized to video frames.
No standalone tool or memo files were saved in this episode. The original session and workspace evidence remain available.
'}
${evidenceCard('Execution evidence','Original verdict, protocol and runtime identity.',[['verdict','Native verdict'],['protocol','Protocol'],['provenance','Source identity']],ep)}${evidenceCard('Session & interactions','Original tool calls, observations and edited workspace.',[['native_session','Native session'],['trace','Trace'],['workspace_changes','Workspace changes']],ep)}
`;
}
async function fetchText(url) {
if(fileCache.has(url))return fileCache.get(url);
const r=await fetch(url);if(!r.ok)throw Error('HTTP '+r.status);
if(Number(r.headers.get('content-length'))>3e6)throw Error('File exceeds the 3 MB preview limit; use Open file');
const text=await r.text();if(text.length>3e6)throw Error('File exceeds the preview limit; use Open file');
fileCache.set(url,text);if(fileCache.size>20)fileCache.delete(fileCache.keys().next().value);return text;
}
async function loadResource(index,token) {
const r=current.ep?.resources?.[index];if(!r)return;
current.resource=index;$('#resource-name').textContent=r.name;$('#resource-open').href=r.file;$('#resource-kind').textContent=r.kind||'Final workspace snapshot';$('#resource-preview').textContent='Loading file…';
$$('[data-resource]').forEach(b=>{b.classList.toggle('active',+b.dataset.resource===index);b.setAttribute('aria-pressed',+b.dataset.resource===index);});
try{const text=await fetchText(r.file);if(token===epoch&¤t.resource===index){$('#resource-preview').textContent=text;current.resourceText=text;}}
catch(e){if(token===epoch&¤t.resource===index){$('#resource-preview').textContent='Preview unavailable: '+e.message;current.resourceText=null;}}
}
const evidenceGroups=[
['Native evaluation','Owner verdicts, measured task and control protocol.',[['verdict','Native verdict'],['timing','Episode timing'],['protocol','Measured protocol'],['native_goal','Task instructions'],['owner_journal','Owner action journal']]],
['Session & workspace','The original agent trajectory and final files.',[['native_session','Native session'],['transcript','Readable transcript'],['trace','Agent trace'],['workspace','Workspace archive'],['workspace_changes','Workspace manifest']]],
['Recording & review','Published media and checks against the recorded evidence.',[['native_video','Original head recording'],['video','Video'],['poster','Poster'],['terminal_frame','Terminal frame'],['recording_manifest','Recording manifest'],['media_check','Media validation'],['analysis','Episode analysis']]],
['Usage & provenance','Provider counters and the actual code used by the episode.',[['usage','Token usage'],['provider_usage','Provider usage ledger'],['provenance','Source identities'],['tool_errors','Tool errors']]]
];
function evidenceCard(title,description,items,ep) {
return `
${esc(c.call_id)}${c.service_response.inference_s.toFixed(3)} s · ${(c.service_response.waypoints||c.service_response.actions).length} × ${(c.service_response.waypoints||c.service_response.actions)[0].length} actions
Task instruction
${esc(c.request.task)}
${images}
${esc(record.contract.image_preprocessing||"Model images follow the recorded checkpoint preprocessing.")} Additional information reaches the model only when present in the request and rendered language below.
${jsonBlock('Complete model-service request',c.request)}${c.model_input_text!==undefined?jsonBlock('Language passed to predict_action · recorded',c.model_input_text,true):''}${jsonBlock('Backbone message · reconstructed from executed code',c.backbone_messages)}
Model-service output
${c.service_response.waypoints?'Untimed joint waypoints consumed by the recorded caller. Kinex or agent-authored code determines their physical timing.':'The recorded action response consumed by the caller.'} Joint values are absolute radians; grippers use −1 open, +1 closed.
${jsonBlock('Complete service response · full precision',c.service_response)}${c.execution?jsonBlock('Actual execution · target count and world time',{executor:c.planner,...c.execution},true):''}${jsonBlock('Internal diagnostic · denormalized training-order actions',c.internal_diagnostics.denormalized_actions)}${caller?`
Kinex tool call & result
This inference is joined to the recorded tool call and result below. Agent-authored code may consume the model output directly; the language agent receives the displayed tool response.
${jsonBlock('Agent tool call',caller.tool_call,true)}${jsonBlock('Tool result received by the language agent',{...caller.tool_result,parts:textParts(caller.tool_result.parts)},true)}
${c.executed_tool_source?jsonBlock('Agent-authored code at this call',c.executed_tool_source):''}`:'
Standalone execution; no language-agent tool call.
'}Recording coverage & model contract
See the per-run contract below for recorded and reconstructed fields. Original requests and rendered language are retained in the current rich-input run; older runs have different recording coverage.
${jsonBlock('Complete recorded StarVLA response · full precision',c.raw_response)}${jsonBlock('Reordered native actions · full precision',c.service_response.actions)}
Standalone execution; no language-agent session or tool call. Token and API costs cover the language agent only; local GPU cost is not reported.
${jsonBlock('Recording coverage & model contract',r.contract)}
`;
function stats(es) {
return UsageMetrics.stats(es);
}
window.benchmarkStats = stats;
const tokens = e => e && Number.isFinite(e.usage?.input_tokens) && Number.isFinite(e.usage?.output_tokens) ? e.usage.input_tokens+e.usage.output_tokens : null;
function metric(value,label,detail='',attrs='') {return `