| <!doctype html> |
| <html lang="en"> |
| <head> |
| <meta charset="utf-8"> |
| <meta name="viewport" content="width=device-width,initial-scale=1"> |
| <meta name="theme-color" content="#f5f7f7"> |
| <meta name="description" content="Compare DreamZero and OpenWAM forecasts with actual simulator rollouts across shared evaluation worlds and tasks, with sixty-world results."> |
| <title>Canon WAM Rollout Lab</title> |
| <link rel="stylesheet" href="style.css"> |
| <script src="app.js" defer></script> |
| </head> |
| <body> |
| <header class="topbar"><a class="brand" href="./"><span class="brand-mark">dz<span></span></span><span>Canon WAM <span class="brand-secondary">/ Rollout Lab</span></span></a><a class="toplink" href="https://huggingface.co/OneWorld-AI" target="_blank" rel="noreferrer">OneWorld AI <span aria-hidden="true">↗</span></a></header> |
| <main> |
| <section class="intro"> |
| <div><div class="eyebrow"><span class="status-dot"></span> ABC SIMULATOR · CANON WAM CHECKPOINTS & TASKS</div><h1>What the model imagined.<br><span>What the robot did.</span></h1><p class="lede">Explore generated futures beside the recorded simulation.<br>Ten shared worlds per checkpoint and task. Every prediction window. <a href="#results">Sixty-world results ↓</a></p></div> |
| <div class="intro-side"><label for="checkpoint">CHECKPOINT / TASK</label><select id="checkpoint" aria-label="Checkpoint"><option value="2000">Step 2,000</option><option value="1000">Step 1,000</option></select><span class="task-label" id="taskLabel">Throw plastic bottles into the bin</span><span id="checkpointProgress" class="checkpoint-progress"></span><div class="tag-row" id="tagRow"><span>Absolute EEF</span><span>Blender</span><span>48 actions / call</span></div></div> |
| </section> |
| <section class="stats" aria-label="Checkpoint results"><div><span class="stat-label">TASK SUCCESS</span><strong id="successStat">— <small>/ 10 worlds</small></strong></div><div><span class="stat-label">MEAN MAX REWARD</span><strong id="rewardStat">—</strong></div><div><span class="stat-label">BUDGET / WORLD</span><strong>22,000 <small>actions</small></strong></div><div><span class="stat-label">SAVED FORECASTS</span><strong id="forecastStat">—</strong></div><div class="stats-note" id="successNote">Success requires all four bottles in the bin. Partial reward shows progress toward that goal.</div></section> |
| <section class="world-section" aria-labelledby="worldsTitle"><div class="section-heading"><div><span class="eyebrow">01 / CHOOSE A WORLD</span><h2 id="worldsTitle">The same scenes, at each checkpoint.</h2></div><span class="hint">Select a thumbnail to explore</span></div><div id="worlds" class="worlds" role="group" aria-label="Evaluation worlds"></div></section> |
| <section id="viewer" class="viewer" aria-labelledby="viewerTitle"> |
| <div class="viewer-heading"><div><div class="eyebrow">02 / COMPARE OBSERVATION & FORECAST</div><h2 id="viewerTitle">World <span id="worldNumber">01</span> <span class="seed">SEED <span id="seedLabel">—</span></span></h2></div><button id="share" class="button quiet" title="Copy a link to this world and timeline position">Copy link <span aria-hidden="true">↗</span></button></div> |
| <div class="players"> |
| <article class="player-card"><div class="player-heading"><div><span class="panel-dot sim"></span><strong>Simulator rollout</strong><span class="panel-subtitle">Recorded observation</span></div><select id="layout" aria-label="Simulator camera layout"><option value="canvas">Matched canvas</option><option value="strip">3-camera strip</option></select></div><div class="video-stage" id="simStage"><video id="simVideo" muted playsinline preload="metadata" aria-label="Recorded simulator rollout"></video><div class="video-watermark">SIMULATOR</div><span class="loading-label" id="simLoading">Loading observation…</span></div><div class="player-footer"><span id="sampleLabel">Recorded sample · step 0</span><a id="simDownload" download target="_blank" rel="noreferrer">Download original ↗</a></div></article> |
| <article class="player-card"><div class="player-heading"><div><span class="panel-dot wam"></span><strong>WAM prediction</strong><span class="panel-subtitle">Generated future</span></div><button class="button mini" id="loopPrediction">▶ Loop this forecast</button></div><div class="video-stage"><video id="wamVideo" muted playsinline preload="metadata" aria-label="Generated WAM prediction"></video><div class="video-watermark amber">GENERATED</div><span class="window-badge" id="windowBadge">FORECAST 001 / 459</span><span class="loading-label" id="wamLoading">Loading prediction…</span></div><div class="player-footer"><span id="forecastLabel">Forecast · steps 0–48</span><a id="wamDownload" download target="_blank" rel="noreferrer">Download all forecasts ↗</a></div></article> |
| </div> |
| <div class="transport"><button id="play" class="button primary" aria-label="Play simulator rollout">▶ Play rollout</button><label class="speed-label">Speed <select id="speed" aria-label="Playback speed in simulated real time"><option value="1">1×</option><option value="4" selected>4×</option><option value="12">12×</option><option value="30">30×</option></select></label><div class="transport-time"><strong id="stepLabel">0 <span>/ 22,000 actions</span></strong><span id="timeLabel">0:00 / 12:28 simulated</span></div><button id="restart" class="button quiet" title="Return to the start">↺ Restart</button></div> |
| <div class="scrubber"><label class="sr-only" for="timeline">Simulator timeline, in actions</label><input id="timeline" type="range" min="0" max="22000" step="1" value="0"><div class="timeline-labels"><span>0</span><span>5,500</span><span>11,000</span><span>16,500</span><span>22,000 actions</span></div></div> |
| <div class="forecast-controls"><div><span class="control-label">INSPECT A FORECAST</span><button id="prevPrediction" class="icon-button" aria-label="Previous forecast">←</button><input id="predictionNumber" type="number" min="1" max="459" value="1" aria-label="Forecast number"><span class="muted">of <span id="predictionCount">459</span></span><button id="nextPrediction" class="icon-button" aria-label="Next forecast">→</button></div><div class="forecast-position" id="forecastPosition">Current frame <span>→ +6 → +12 → … → +48 actions</span></div><button id="firstReward" class="button quiet" disabled>Jump to first reward ↗</button></div> |
| <p class="sampling-note" id="samplingNote">Loading recording cadence…</p> |
| <div class="world-result"><span id="outcomeBadge" class="outcome">Budget reached</span><span>Selected world: max reward <strong id="worldMax">—</strong> · final reward <strong id="worldFinal">—</strong></span><a id="dataDownload" download target="_blank" rel="noreferrer">World data & provenance ↗</a></div> |
| </section> |
| <section class="reward-section" aria-labelledby="rewardTitle"><div class="section-heading"><div><span class="eyebrow">03 / FOLLOW THE OUTCOME</span><h2 id="rewardTitle">Reward over the rollout.</h2></div><div class="chart-legend" id="chartLegend"><span><i class="legend-line cp2000"></i>Step 2,000</span><span><i class="legend-line cp1000"></i>Step 1,000</span></div></div><div class="chart-wrap"><svg id="rewardChart" viewBox="0 0 1200 190" role="img" aria-label="Reward against simulator actions for available checkpoints"></svg><p>Click the chart to seek. Reward is measured every 50 actions. A value of 1 means the task is solved.</p></div></section> |
| <section id="results" class="results" aria-labelledby="resultsTitle"><div class="section-heading"><div><span class="eyebrow">04 / SIXTY-WORLD RESULTS</span><h2 id="resultsTitle">How the 14B OpenWAM compares.</h2></div><span class="hint">Same seeds for every model · Blender · 22,000-action budget</span></div> |
| <div class="results-grid"> |
| <div class="result-card"><h3>Throw plastic bottles into the bin <small>60 worlds</small></h3><div class="table-scroll"><table><thead><tr><th>Model</th><th>Actions / call</th><th>Solved</th><th>Mean final reward</th><th>Median steps to solve</th></tr></thead><tbody> |
| <tr class="ours"><td>OpenWAM canon 14B · step 57,000 (EMA)</td><td>32</td><td>41 / 60</td><td>0.900</td><td>1,550</td></tr> |
| <tr><td>pi0.5</td><td>15</td><td>44 / 60</td><td>0.917</td><td>3,050</td></tr> |
| <tr><td>DreamZero · step 18,000</td><td>48</td><td>44 / 60</td><td>0.917</td><td>1,925</td></tr> |
| <tr><td>ABC-DiT</td><td>30</td><td>33 / 60</td><td>0.863</td><td>4,200</td></tr> |
| <tr><td>OpenWAM-α · step 22,000</td><td>32</td><td>6 / 60</td><td>0.533</td><td>5,900</td></tr> |
| <tr><td>DreamZero canon · step 2,000</td><td>48</td><td>0 / 60</td><td>0.029*</td><td>–</td></tr> |
| </tbody></table></div><p class="table-note">Paired by world, the 14B model is not distinguishable from pi0.5 (12 vs 15 worlds solved only by one model, p = 0.70) or DreamZero 18k (8 vs 11, p = 0.65). 18 of its 19 misses stop at 0.75: the last bottle never reaches the bin. *Mean max reward.</p></div> |
| <div class="result-card"><h3>Experiments 6–9 tasks <small>60 worlds each · solved, mean final reward</small></h3><div class="table-scroll"><table><thead><tr><th>Task</th><th>OpenWAM canon 14B</th><th>OpenWAM-α · step 24,000</th><th>DreamZero · step 18,000</th><th>pi0.5 task-finetune · step 94,000</th></tr></thead><tbody> |
| <tr><td>Turn the mug right side up</td><td class="ours">20 / 60 · 0.519</td><td>17 / 60 · 0.403</td><td>1 / 60 · 0.033</td><td>19 / 60 · 0.458</td></tr> |
| <tr><td>Sweep away the paper scraps</td><td class="ours">10 / 60 · 0.381</td><td>0 / 60 · 0.011</td><td>1 / 60 · 0.108</td><td>0 / 60 · 0.036</td></tr> |
| <tr><td>Load the plates into the dish rack</td><td class="ours">3 / 60 · 0.113</td><td>2 / 60 · 0.042</td><td>2 / 60 · 0.071</td><td>1 / 60 · 0.031</td></tr> |
| <tr><td>Set up the chess pieces</td><td class="ours">0 / 56 · 0.004</td><td>0 / 50 · 0.000</td><td>1 / 57 · 0.018</td><td>0 / 58 · 0.000</td></tr> |
| </tbody></table></div><p class="table-note">Sweep is the clear gain: the 14B model solves 10 worlds the others miss, with 0 the other way against OpenWAM-α (p = 0.002) and 1 against DreamZero (p = 0.01). Mug flip is level with OpenWAM-α (11 vs 8, p = 0.65). All 10 sweep successes end with the bin upright and the table clear. pi0.5 (15 actions per call) matches the 14B model on mug flip (9 vs 10 worlds solved only by one model, p = 1) but solves no sweep worlds (0 vs 10, p = 0.002). In 2 of the 3 14B plate successes the plate is still in the gripper at the rack edge, and the single pi0.5 plate success is a plate standing against the outside of the rack, so no model reliably solves plates. Chess counts leave out worlds that crashed the physics engine (4, 5, 3 and 2 for pi0.5); OpenWAM-α also has 5 unfinished chess worlds.</p></div> |
| <div class="result-card"><h3>Six more tasks <small>14B OpenWAM only · 10 worlds each</small></h3><div class="table-scroll"><table><thead><tr><th>Task</th><th>Solved</th><th>Mean final reward</th><th>Mean max reward</th></tr></thead><tbody> |
| <tr><td>Pick from the conveyor</td><td>3 / 10</td><td>0.567</td><td>0.567</td></tr> |
| <tr><td>Sort the Lego blocks</td><td>1 / 10</td><td>0.891</td><td>0.902</td></tr> |
| <tr><td>Hand an item to the other side</td><td>1 / 10</td><td>0.100</td><td>0.300</td></tr> |
| <tr><td>Sort the nuts and bolts</td><td>0 / 10</td><td>0.026</td><td>0.148</td></tr> |
| <tr><td>Pour the beads</td><td>0 / 10</td><td>0.000</td><td>0.022</td></tr> |
| <tr class="flagged"><td>Hang the mug on the mug rack</td><td>(8 / 10)</td><td>(0.850)</td><td>(0.917)</td></tr> |
| </tbody></table></div><p class="table-note">The mug-rack scorer is unreliable: it also counts a mug still held in the gripper next to the rack, or one standing at its base, so that row is not a valid success rate. On Lego, most worlds finish one block short.</p></div> |
| </div> |
| </section> |
| <details class="method"><summary>About these recordings <span>Protocol, timing & interpretation</span></summary><div class="method-grid"><div><h3>Closed-loop simulation</h3><p>Each call sees the latest simulator cameras and measured state, predicts an action block (48 actions for DreamZero, 32 for OpenWAM 14B), and executes all of it. Generated video is saved for inspection and is never fed back as the next observation.</p></div><div><h3>Independent prediction windows</h3><p>Every forecast contains the current frame followed by eight generated frames. “Loop this forecast” isolates one window. “Play rollout” follows the full simulator timeline and selects its corresponding forecast.</p></div><div><h3>Matching the training contract</h3><p id="contractNote">Absolute canonical base-frame end-effector actions, ABC-source quantile normalization, canonical grippers, and inverse kinematics. The canvas puts the top camera above the left and right wrist cameras.</p></div><div><h3>Read results at the right scope</h3><p>Each checkpoint or task in the viewer shows ten worlds; the sixty-world tables above use the full evaluations. New recordings appear individually as they finish; the ready count shows the scope of the displayed results. All 22,000 actions are budgeted; the last stored simulator image is at action <span id="lastSampleLabel">—</span>.</p></div></div></details> |
| <footer><span>OneWorld AI · DreamZero (exp 10) · OpenWAM 14B (exp 12)</span><a href="manifest.json" target="_blank" rel="noreferrer">Evaluation manifest ↗</a><span id="buildLabel"></span></footer> |
| <div id="toast" class="toast" role="status" aria-live="polite"></div> |
| <div id="error" class="error" role="alert" hidden></div> |
| </main> |
| </body> |
| </html> |
|
|
