| <!doctype html> |
| <html lang="en"> |
| <head> |
| <meta charset="utf-8"> |
| <meta name="viewport" content="width=device-width, initial-scale=1"> |
| <title>PACK3D — The repacking demo</title> |
| <meta name="description" content="Explore why geometry matters in packed 3D attention. Repack the same volume and compare PACK3D with sequence-indexed RoPE, using measured Allen-15 results."> |
| <meta name="theme-color" content="#F4F4F4"> |
| <link rel="icon" href="favicon.svg" type="image/svg+xml"> |
| <link rel="preload" href="inter-latin-standard-normal.woff2" as="font" type="font/woff2" crossorigin> |
| <link rel="stylesheet" href="stage.css"> |
| <link rel="stylesheet" href="style.css"> |
| <script src="app.js" defer></script> |
| <noscript><style>.sw-canvas, .sw-n, .sw-foot { display: none; } .sw-fallback { display: block; } .sw-cells { aspect-ratio: auto; }</style></noscript> |
| </head> |
| <body> |
| <a class="skip" href="#demo">Skip to the demo</a> |
| <header class="top"> |
| <a class="brand" href="https://amilworks.github.io/Pack3D/" target="_blank" rel="noopener" aria-label="PACK3D project website (opens in a new tab)"> |
| <picture><source srcset="logo-dark.svg" media="(prefers-color-scheme: dark)"><img src="logo.svg" alt="PACK3D" width="100" height="38"></picture> |
| </a> |
| <nav aria-label="Project links"> |
| <a href="https://amilworks.github.io/Pack3D/" target="_blank" rel="noopener">Website<span class="sr-only"> (opens in a new tab)</span></a> |
| <a href="https://github.com/amilworks/Pack3D" target="_blank" rel="noopener">GitHub<span class="sr-only"> (opens in a new tab)</span></a> |
| <a class="paper-link" href="https://arxiv.org/abs/2610.38694" target="_blank" rel="noopener">Paper<span class="sr-only"> (opens in a new tab)</span></a> |
| </nav> |
| </header> |
| <main id="demo"> |
| <section class="intro" aria-labelledby="title"> |
| <p class="eyebrow">Geometry Matters in Packed 3D Attention</p> |
| <h1>Change the memory. Keep the geometry.</h1> |
| <p>A volume has coordinates. Its tokens also have positions in memory. Move the tokens, keeping their geometry intact, and see how two attention models respond.</p> |
| </section> |
|
|
| <section class="stage-window" id="repackStage" aria-labelledby="stage-title" aria-describedby="demo-note"> |
| <div class="sw-in"> |
| <div class="sw-head"> |
| <p class="eyebrow">The repacking demo</p> |
| <h2 class="sw-title" id="stage-title">Same cell. Same weights. A new memory order.</h2> |
| </div> |
| <div class="sw-cells"> |
| <canvas class="sw-canvas" id="swCanvas" tabindex="0" role="application" aria-roledescription="3D view" aria-label="Two illustrative cells under the same repack." aria-describedby="demo-note"></canvas> |
| <div id="swA" aria-hidden="true"></div><div id="swB" aria-hidden="true"></div> |
| <p class="sw-fallback">Under the repack, about 18 of 262,144 predictions change for PACK3D and about 3,215 for packed RoPE.</p> |
| </div> |
| <div class="sw-under"> |
| <div class="sw-col"> |
| <p class="sw-who"><span class="dot teal"></span>PACK3D <small>uses grid coordinates</small></p> |
| <div class="sw-strip"><canvas id="swStripA" aria-hidden="true"></canvas></div> |
| <p class="sw-n"><span class="sw-big sw-teal" id="swNA">0</span><span class="sw-lab">predictions changed<br><span id="swRA">0.0069%</span> of positions</span></p> |
| </div> |
| <div class="sw-col"> |
| <p class="sw-who"><span class="dot coral"></span>Packed RoPE <small>uses memory positions</small></p> |
| <div class="sw-strip"><canvas id="swStripB" aria-hidden="true"></canvas></div> |
| <p class="sw-n"><span class="sw-big sw-coral" id="swNB">0</span><span class="sw-lab">predictions changed<br><span id="swRB">1.23%</span> of positions</span></p> |
| </div> |
| </div> |
| <div class="sw-foot"> |
| <div class="sw-act"><button class="btn" id="swGo" type="button">Repack the memory</button><button class="btn plain" type="button" data-motion>Pause motion</button></div> |
| <p class="sw-state" id="swState"><i aria-hidden="true"></i><span class="sw-state-t"><span id="swStateT">Training layout: raster order</span><span class="sw-hold" aria-hidden="true"></span><span class="sw-hold" aria-hidden="true"></span></span><span class="sr-only" id="swStateX"></span></p> |
| </div> |
| </div> |
| </section> |
| <p class="gesture">Drag a cell, swipe sideways, or focus the view and use the arrow keys to turn both.</p> |
| <p class="disclosure" id="demo-note"><strong>An illustration of measured results.</strong> Counts are the mean changed-prediction rates across six seeds, scaled to 262,144 positions. The cell and the locations of the changes are illustrative; this Space does not run a trained model. The memory strips show sixteen groups moved whole.</p> |
| <noscript><p class="disclosure">Enable JavaScript to interact. The measured rates correspond to about 18 changed predictions for PACK3D and 3,215 for packed RoPE per 262,144 positions. The results below remain available without JavaScript.</p></noscript> |
|
|
| <section class="results" aria-labelledby="results-title"> |
| <div class="result-copy"> |
| <p class="eyebrow">The measured result</p> |
| <h2 id="results-title">Does the prediction follow the object?</h2> |
| <p>Both models reach similar accuracy on the training layout. After a pure repack, PACK3D stays nearly unchanged; packed RoPE loses Dice in all six seeds.</p> |
| <a href="https://github.com/amilworks/Pack3D/blob/main/results/allen15_repacking/seed_metrics.csv" target="_blank" rel="noopener">View the per-seed results ↗<span class="sr-only"> (opens in a new tab)</span></a> |
| </div> |
| <div class="result-table"> |
| <table> |
| <caption>Allen-15 · mean validation macro Dice</caption> |
| <thead><tr><th scope="col">Model</th><th scope="col">Training<br>layout</th><th scope="col">Pure<br>repack</th></tr></thead> |
| <tbody><tr><th scope="row">PACK3D</th><td id="ax0">0.8247</td><td id="ax1">0.8248</td></tr><tr><th scope="row">Packed RoPE</th><td id="pk0">0.8229</td><td id="pk1">0.7750</td></tr></tbody> |
| </table> |
| <p class="table-note">Six matched seeds · 120 validation crops · same checkpoints, no retraining. Values from the released camera-ready results.</p> |
| </div> |
| </section> |
|
|
| <section class="learn" aria-labelledby="learn-title"> |
| <div><p class="eyebrow">Try it in code</p><h2 id="learn-title">Run a real equivariance check.</h2></div> |
| <div><p>A pure repack carries each token’s coordinate, neighbors, summary memberships, and within-block slot along with it. The CPU example checks aligned outputs of a randomly initialized attention layer. It is separate from the trained-model results above.</p><a class="text-link" href="https://github.com/amilworks/Pack3D/blob/main/examples/repack_test.py" target="_blank" rel="noopener">Run the CPU test ↗<span class="sr-only"> (opens in a new tab)</span></a></div> |
| </section> |
| </main> |
| <footer><p>Amil Khan · Anantajit Subrahmanya · B.S. Manjunath<br>University of California, Santa Barbara · NeurIPS 2026</p><a href="https://amilworks.github.io/Pack3D/#cite" target="_blank" rel="noopener">Citation ↗<span class="sr-only"> (opens in a new tab)</span></a></footer> |
| </body> |
| </html> |
|
|
