Reuse input video for RGB previews and remove redundant viewer chrome
Browse files- DEPLOYMENT.md +17 -5
- README.md +12 -2
- app.py +4 -2
- deployment-manifest.json +11 -10
- space.css +2 -6
- viewer/app.js +16 -11
- viewer/frame-image-cache.js +74 -36
- viewer/index.html +1 -5
- viewer/rgb-preview.js +101 -0
- viewer/style.css +3 -16
DEPLOYMENT.md
CHANGED
|
@@ -115,7 +115,7 @@ replaces the website's Hugging Face button and says **see more demos!**;
|
|
| 115 |
there is no Model weights link in the navigation. GitHub and GameRealm keep
|
| 116 |
the website's targets. The website has no technical-report URL configured,
|
| 117 |
so the report label is explicitly disabled rather than opening a broken link.
|
| 118 |
-
Explore demos
|
| 119 |
Standalone, font-independent copies of the wordmark are available in
|
| 120 |
`assets/branding/pi4-gradient.svg` and `assets/branding/pi4-black.svg` for
|
| 121 |
sharing or embedding outside the Space. Their paths match the website mark.
|
|
@@ -151,10 +151,22 @@ The viewer prefetches 32 future frames with at most three concurrent bulk
|
|
| 151 |
downloads, sharing those slots with background track loading. Point chunks
|
| 152 |
are deduplicated, prioritized for the current selection and applied in small
|
| 153 |
cooperative batches. Failed speculative requests retry only when needed.
|
| 154 |
-
Keep `viewer/frame-image-cache.js`
|
| 155 |
-
|
| 156 |
-
|
| 157 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 158 |
|
| 159 |
The saved `tracks.bin` loads independently with a 120-second timeout and at
|
| 160 |
most three attempts for timeouts, network errors, HTTP 408/429, or HTTP 5xx.
|
|
|
|
| 115 |
there is no Model weights link in the navigation. GitHub and GameRealm keep
|
| 116 |
the website's targets. The website has no technical-report URL configured,
|
| 117 |
so the report label is explicitly disabled rather than opening a broken link.
|
| 118 |
+
The extra Explore demos button is omitted; the footer has no extra button.
|
| 119 |
Standalone, font-independent copies of the wordmark are available in
|
| 120 |
`assets/branding/pi4-gradient.svg` and `assets/branding/pi4-black.svg` for
|
| 121 |
sharing or embedding outside the Space. Their paths match the website mark.
|
|
|
|
| 151 |
downloads, sharing those slots with background track loading. Point chunks
|
| 152 |
are deduplicated, prioritized for the current selection and applied in small
|
| 153 |
cooperative batches. Failed speculative requests retry only when needed.
|
| 154 |
+
Keep `viewer/frame-image-cache.js` and `viewer/rgb-preview.js` with the other
|
| 155 |
+
viewer assets. The embedded RGB inset reuses the exact matching input video
|
| 156 |
+
URL from the parent player, synchronized to `source_timestamps_s`. A separate
|
| 157 |
+
muted decoder leaves the input player's playback and seek position untouched;
|
| 158 |
+
already-ready matching input frames can be drawn directly. The result iframe
|
| 159 |
+
binds the video's expected path at reconstruction time, so changing the upload
|
| 160 |
+
cannot attach the wrong video. Missing metadata, new tabs, image inputs,
|
| 161 |
+
video failures, and a stalled decoder fall back to exported RGB images.
|
| 162 |
+
The persistent preview canvas only paints successfully decoded frames and
|
| 163 |
+
keeps the last valid image on error. Fallback images share requests with the
|
| 164 |
+
filmstrip, prefetch 24 ahead and four behind with three concurrent requests,
|
| 165 |
+
retry failures up to three times, retain encoded bytes for revisits, and cap
|
| 166 |
+
decoded images at 32. Leaving the viewer aborts requests and releases video
|
| 167 |
+
and image resources. The redundant inner branding/session bar is removed;
|
| 168 |
+
the viewer uses the full iframe height. None of these changes reduce the
|
| 169 |
+
point budget or alter inference.
|
| 170 |
|
| 171 |
The saved `tracks.bin` loads independently with a 120-second timeout and at
|
| 172 |
most three attempts for timeouts, network errors, HTTP 408/429, or HTTP 5xx.
|
README.md
CHANGED
|
@@ -30,7 +30,7 @@ lockup, pill buttons, SVG icons, and local Caveat handwritten callouts. The
|
|
| 30 |
website's Hugging Face button becomes **Project Page**, annotated **see more
|
| 31 |
demos!**; no Model weights button is shown. **GitHub** and **GameRealm** retain
|
| 32 |
their website links. The technical report stays disabled until its URL is
|
| 33 |
-
available. **Explore demos**
|
| 34 |
the website branding without an extra button. The full-width white scene viewer
|
| 35 |
sits directly below the navigation. The single-line project title appears inside
|
| 36 |
the empty viewer, above "Your scene, in another dimension." The viewer, input,
|
|
@@ -111,11 +111,21 @@ The viewer prefetches 32 future frames with up to three concurrent bulk
|
|
| 111 |
downloads (including tracks), instead of waiting at each eight-frame boundary.
|
| 112 |
Tracks load in the background and their controls become available when ready;
|
| 113 |
they do not block the initial scene or playback. RGB previews are decoded ahead
|
| 114 |
-
of playback
|
| 115 |
cooperative batches to keep the page responsive. These changes do not lower
|
| 116 |
the point budget or change inference. A connection slower than the scene's
|
| 117 |
required data rate can still need buffering.
|
| 118 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 119 |
Trajectory downloads use a separate two-minute timeout and retry temporary
|
| 120 |
network failures up to three attempts. The layer panel shows loading/error
|
| 121 |
status and a **Retry tracks** button if needed; it reloads the saved trajectory
|
|
|
|
| 30 |
website's Hugging Face button becomes **Project Page**, annotated **see more
|
| 31 |
demos!**; no Model weights button is shown. **GitHub** and **GameRealm** retain
|
| 32 |
their website links. The technical report stays disabled until its URL is
|
| 33 |
+
available. The extra **Explore demos** button is omitted. The footer retains
|
| 34 |
the website branding without an extra button. The full-width white scene viewer
|
| 35 |
sits directly below the navigation. The single-line project title appears inside
|
| 36 |
the empty viewer, above "Your scene, in another dimension." The viewer, input,
|
|
|
|
| 111 |
downloads (including tracks), instead of waiting at each eight-frame boundary.
|
| 112 |
Tracks load in the background and their controls become available when ready;
|
| 113 |
they do not block the initial scene or playback. RGB previews are decoded ahead
|
| 114 |
+
of playback when image fallback is needed, and point chunks are applied in small
|
| 115 |
cooperative batches to keep the page responsive. These changes do not lower
|
| 116 |
the point budget or change inference. A connection slower than the scene's
|
| 117 |
required data rate can still need buffering.
|
| 118 |
|
| 119 |
+
The RGB inset reuses the matching input video's URL and sampled source
|
| 120 |
+
timestamps when embedded in the Space, without seeking or changing the input
|
| 121 |
+
player itself. It draws onto a persistent canvas rather than swapping image
|
| 122 |
+
elements. Image uploads, standalone viewers, changed inputs, and video errors
|
| 123 |
+
use the exported RGB images instead. These images share requests with the
|
| 124 |
+
filmstrip, retain downloaded bytes for revisits, and cap the image cache at 32
|
| 125 |
+
decoded frames. Temporary failures retry up to three times; failures never replace a
|
| 126 |
+
valid preview with a broken image. The redundant viewer branding/session bar
|
| 127 |
+
is removed so the scene uses the iframe height.
|
| 128 |
+
|
| 129 |
Trajectory downloads use a separate two-minute timeout and retry temporary
|
| 130 |
network failures up to three attempts. The layer panel shows loading/error
|
| 131 |
status and a **Retry tracks** button if needed; it reloads the saved trajectory
|
app.py
CHANGED
|
@@ -1,5 +1,6 @@
|
|
| 1 |
import gc
|
| 2 |
from functools import lru_cache, partial
|
|
|
|
| 3 |
import json
|
| 4 |
import logging
|
| 5 |
import math
|
|
@@ -10,6 +11,7 @@ import shutil
|
|
| 10 |
import sys
|
| 11 |
import time
|
| 12 |
import uuid
|
|
|
|
| 13 |
|
| 14 |
import spaces
|
| 15 |
import gradio as gr
|
|
@@ -60,7 +62,6 @@ HEADER = """
|
|
| 60 |
<span class="nav-button" aria-disabled="true" title="Technical report link coming soon"><svg aria-hidden="true"><use href="#space-icon-report"/></svg>Technical Report</span>
|
| 61 |
<span class="nav-item"><span class="nav-note nav-note-right" aria-hidden="true"><span>see more demos!</span><svg viewBox="0 0 28 22"><path d="M3 2c10 0 17 5 18 14m-6-4 6 5 4-7"/></svg></span><a class="nav-button nav-external" href="https://yyfz.github.io/pi4" target="_blank" rel="noopener"><svg aria-hidden="true"><use href="#space-icon-external"/></svg>Project Page</a></span>
|
| 62 |
<span class="nav-item"><span class="nav-note nav-note-left" aria-hidden="true"><span>open dataset!</span><svg viewBox="0 0 28 22"><path d="M3 2c10 0 17 5 18 14m-6-4 6 5 4-7"/></svg></span><a class="nav-button nav-external" href="https://github.com/weiguangzhao/GameRealm.github.io" target="_blank" rel="noopener"><svg aria-hidden="true" viewBox="0 0 24 24"><ellipse cx="12" cy="5" rx="8" ry="3"/><path d="M4 5v7c0 1.7 3.6 3 8 3s8-1.3 8-3V5M4 12v7c0 1.7 3.6 3 8 3s8-1.3 8-3v-7"/></svg>GameRealm</a></span>
|
| 63 |
-
<a class="nav-button nav-demo" href="#pi4-examples-card">Explore demos<svg class="demo-arrow" aria-hidden="true"><use href="#space-icon-arrow"/></svg></a>
|
| 64 |
</nav>
|
| 65 |
</header>
|
| 66 |
"""
|
|
@@ -277,9 +278,10 @@ def reconstruct(
|
|
| 277 |
"One joint forward pass · Results kept for up to 6 hours."
|
| 278 |
)
|
| 279 |
url = f"/view/{run.name}/"
|
|
|
|
| 280 |
viewer = (
|
| 281 |
f'<div class="viewer-link"><a href="{url}" target="_blank" rel="noopener">Open in a new tab ↗</a></div>'
|
| 282 |
-
f'<iframe src="{url}" title="Pi4 geometry and motion viewer" '
|
| 283 |
'style="width:100%;height:720px;border:0;display:block" '
|
| 284 |
'allow="fullscreen" allowfullscreen></iframe>'
|
| 285 |
)
|
|
|
|
| 1 |
import gc
|
| 2 |
from functools import lru_cache, partial
|
| 3 |
+
from html import escape
|
| 4 |
import json
|
| 5 |
import logging
|
| 6 |
import math
|
|
|
|
| 11 |
import sys
|
| 12 |
import time
|
| 13 |
import uuid
|
| 14 |
+
from urllib.parse import quote
|
| 15 |
|
| 16 |
import spaces
|
| 17 |
import gradio as gr
|
|
|
|
| 62 |
<span class="nav-button" aria-disabled="true" title="Technical report link coming soon"><svg aria-hidden="true"><use href="#space-icon-report"/></svg>Technical Report</span>
|
| 63 |
<span class="nav-item"><span class="nav-note nav-note-right" aria-hidden="true"><span>see more demos!</span><svg viewBox="0 0 28 22"><path d="M3 2c10 0 17 5 18 14m-6-4 6 5 4-7"/></svg></span><a class="nav-button nav-external" href="https://yyfz.github.io/pi4" target="_blank" rel="noopener"><svg aria-hidden="true"><use href="#space-icon-external"/></svg>Project Page</a></span>
|
| 64 |
<span class="nav-item"><span class="nav-note nav-note-left" aria-hidden="true"><span>open dataset!</span><svg viewBox="0 0 28 22"><path d="M3 2c10 0 17 5 18 14m-6-4 6 5 4-7"/></svg></span><a class="nav-button nav-external" href="https://github.com/weiguangzhao/GameRealm.github.io" target="_blank" rel="noopener"><svg aria-hidden="true" viewBox="0 0 24 24"><ellipse cx="12" cy="5" rx="8" ry="3"/><path d="M4 5v7c0 1.7 3.6 3 8 3s8-1.3 8-3V5M4 12v7c0 1.7 3.6 3 8 3s8-1.3 8-3v-7"/></svg>GameRealm</a></span>
|
|
|
|
| 65 |
</nav>
|
| 66 |
</header>
|
| 67 |
"""
|
|
|
|
| 278 |
"One joint forward pass · Results kept for up to 6 hours."
|
| 279 |
)
|
| 280 |
url = f"/view/{run.name}/"
|
| 281 |
+
preview_source = "" if input_type == "Images" else escape(f"/gradio_api/file={quote(str(source.resolve()), safe='/')}", quote=True)
|
| 282 |
viewer = (
|
| 283 |
f'<div class="viewer-link"><a href="{url}" target="_blank" rel="noopener">Open in a new tab ↗</a></div>'
|
| 284 |
+
f'<iframe src="{url}" data-source-video="{preview_source}" title="Pi4 geometry and motion viewer" '
|
| 285 |
'style="width:100%;height:720px;border:0;display:block" '
|
| 286 |
'allow="fullscreen" allowfullscreen></iframe>'
|
| 287 |
)
|
deployment-manifest.json
CHANGED
|
@@ -4,16 +4,16 @@
|
|
| 4 |
"team": "Physical Intelligence Team",
|
| 5 |
"source_space_revision": "989f037ec366dedae7b81c59a45a14f861ff3599",
|
| 6 |
"target_space_repository": "yyfz233/Pi4",
|
| 7 |
-
"note": "Initially copied from the committed source Space, excluding outputs, caches, and Git metadata. Hashes describe the current target Space working tree, including Space-only follow-up fixes, synchronized compressed example videos, generic Figure Skating naming, the six-example gallery with three additional project-website RGB inputs and a shared example catalog, website-matched vector branding, navigation, header/footer AI Lab logo, pill buttons and handwritten Project Page callouts with locally bundled Inter, Gilda Display and Caveat fonts, a compact layout with an in-viewer single-line title, no numbered card heading bars or footer button, and a 1.5-second default track trail, and exclude this manifest.",
|
| 8 |
"sha256": {
|
| 9 |
".gitattributes": "1ab9179cacb2209b7946a7ed89a4700db8464ce3381942d8688c4c90594e195a",
|
| 10 |
".gitignore": "cf8476943466af0150f939f03666befd77c7c2b511e14f4d5ea32b37093816ca",
|
| 11 |
-
"DEPLOYMENT.md": "
|
| 12 |
"LICENSE": "a27c27dfee488645aa7f6b611083b6e9b151feca14f3e4b603dd1a3076a3deb1",
|
| 13 |
"LICENSE_MODEL": "3fbf2bec464d033c4a71796fe05068f3f6ccc8e8bf04cd283f501b3ff2cde0dc",
|
| 14 |
"NOTICE": "cdf0667d10d6c932d2c9fb6b0f6608d0cfeea2b9af0ad8a3bd8b5f46cd5b3fa8",
|
| 15 |
-
"README.md": "
|
| 16 |
-
"app.py": "
|
| 17 |
"assets/branding/README.md": "83a2cc2a5ebf9219f5f2f043f9dc266e1ff7c6156d3d43f66a32aadf9f912d6b",
|
| 18 |
"assets/branding/ailab-logo-transparent.webp": "6e2f385b6aae394b29b4a4a6f303875081bb77355f138fe58929ca48e08ac671",
|
| 19 |
"assets/branding/pi4-black.svg": "68530011046edf4ab6c8c445be2b288f664d55caa9a104805a418d5b2ef68169",
|
|
@@ -60,25 +60,26 @@
|
|
| 60 |
"pyproject.toml": "fe163ea0b981e41ddcd6d25ce9ce49e1548dc6184f0eab80077c35701eb76ae2",
|
| 61 |
"requirements.txt": "ec5668859d3aa420aa9a95448920ff6c082053714a3bad2ada851431f2ea3b49",
|
| 62 |
"scripts/build_example_covers.py": "17e091eddc3bc6749235ea764c998c971b6b457007cd3146b0e6f832c36e54c5",
|
| 63 |
-
"space.css": "
|
| 64 |
-
"viewer/app.js": "
|
| 65 |
"viewer/camera-follow.js": "eb7f768c929f5d5933b167a5380cb308c0a5dca7e047cbff1f858025c1021cd5",
|
| 66 |
"viewer/camera-layer.js": "96c625b0769a2ce8b6826556f781e75d536fb1f65f626c61d9752955981361b0",
|
| 67 |
"viewer/controls.js": "1a4f25294e3b2a72c772aeadadfb9d74e1e8772015764cb0a2b49559864a66d6",
|
| 68 |
"viewer/dynamic-tracks.js": "d676672d0a7359a68d2150289c07db432b071b7ac12d8b566c33240fa9176493",
|
| 69 |
-
"viewer/frame-image-cache.js": "
|
| 70 |
-
"viewer/index.html": "
|
| 71 |
"viewer/point-cloud.js": "9a75bd0b78bdbfb7bfd925422e2b748adaa7e43465a84667e7c7da464deea874",
|
| 72 |
"viewer/point-picking.js": "de7bddd93d60bdd249c3df7b68e98637102125581487e711c1ad4d2a8090512a",
|
| 73 |
"viewer/point-sampling.js": "5915c7a7e67afbe9e59082980276accf44f2a3e6e9d9f6da8759bacbb47e4bd6",
|
|
|
|
| 74 |
"viewer/serve.py": "4b4b51392a4ee4c43c3178ca5520bba5627d442aacc070ab4cc8c2f5b7d22644",
|
| 75 |
"viewer/sort-worker.js": "03f9041725a46b388dffc7bda85d952d412510bca30b5687d5556c6ab7605670",
|
| 76 |
"viewer/static-demo-data.js": "ba8ddf67ff151168c6d97310458241faafc2f53a4763abe34739e0ca66b7e865",
|
| 77 |
"viewer/static-demo-viewer.js": "3b60da852877c49cbbc277bf5803540927600d3c945644e9c193eab4f295246e",
|
| 78 |
-
"viewer/style.css": "
|
| 79 |
"viewer/vendor/TrackballControls.js": "34e097b83260d8f40d23473bc7511889909660f89de9734e46928df9128fd2a8",
|
| 80 |
"viewer/vendor/three.core.min.js": "61ba0df005b05991361d040d8ff670e1aadfd0ce7aeebd1fdb0725957a8957de",
|
| 81 |
"viewer/vendor/three.module.min.js": "e2b5ee6bccd38fd6d8a2428546b83c5f2426d84b152ef82be8055556e3b40eb6"
|
| 82 |
},
|
| 83 |
-
"updated_at": "2026-11-
|
| 84 |
}
|
|
|
|
| 4 |
"team": "Physical Intelligence Team",
|
| 5 |
"source_space_revision": "989f037ec366dedae7b81c59a45a14f861ff3599",
|
| 6 |
"target_space_repository": "yyfz233/Pi4",
|
| 7 |
+
"note": "Initially copied from the committed source Space, excluding outputs, caches, and Git metadata. Hashes describe the current target Space working tree, including Space-only follow-up fixes, synchronized compressed example videos, generic Figure Skating naming, the six-example gallery with three additional project-website RGB inputs and a shared example catalog, website-matched vector branding, navigation, header/footer AI Lab logo, pill buttons and handwritten Project Page callouts with locally bundled Inter, Gilda Display and Caveat fonts, a compact layout with an in-viewer single-line title, no numbered card heading bars or footer button, source-video RGB previews with shared image fallback caching and retry, no redundant inner viewer header or Explore demos button, and a 1.5-second default track trail, and exclude this manifest.",
|
| 8 |
"sha256": {
|
| 9 |
".gitattributes": "1ab9179cacb2209b7946a7ed89a4700db8464ce3381942d8688c4c90594e195a",
|
| 10 |
".gitignore": "cf8476943466af0150f939f03666befd77c7c2b511e14f4d5ea32b37093816ca",
|
| 11 |
+
"DEPLOYMENT.md": "f59125719b0cec77c362b562c077f8f0cf412e607c3ef95737afd8ee6f30d522",
|
| 12 |
"LICENSE": "a27c27dfee488645aa7f6b611083b6e9b151feca14f3e4b603dd1a3076a3deb1",
|
| 13 |
"LICENSE_MODEL": "3fbf2bec464d033c4a71796fe05068f3f6ccc8e8bf04cd283f501b3ff2cde0dc",
|
| 14 |
"NOTICE": "cdf0667d10d6c932d2c9fb6b0f6608d0cfeea2b9af0ad8a3bd8b5f46cd5b3fa8",
|
| 15 |
+
"README.md": "ff5caacb9ea2caf2cf9f8f19c32086eeff5f347c965afd87bb9ef583ff766cb6",
|
| 16 |
+
"app.py": "e8f41807a8a2489fe5e08b0a396e6d9b7cc87c85a563133e42469a271273b805",
|
| 17 |
"assets/branding/README.md": "83a2cc2a5ebf9219f5f2f043f9dc266e1ff7c6156d3d43f66a32aadf9f912d6b",
|
| 18 |
"assets/branding/ailab-logo-transparent.webp": "6e2f385b6aae394b29b4a4a6f303875081bb77355f138fe58929ca48e08ac671",
|
| 19 |
"assets/branding/pi4-black.svg": "68530011046edf4ab6c8c445be2b288f664d55caa9a104805a418d5b2ef68169",
|
|
|
|
| 60 |
"pyproject.toml": "fe163ea0b981e41ddcd6d25ce9ce49e1548dc6184f0eab80077c35701eb76ae2",
|
| 61 |
"requirements.txt": "ec5668859d3aa420aa9a95448920ff6c082053714a3bad2ada851431f2ea3b49",
|
| 62 |
"scripts/build_example_covers.py": "17e091eddc3bc6749235ea764c998c971b6b457007cd3146b0e6f832c36e54c5",
|
| 63 |
+
"space.css": "945de0686bb942400d362dfbb144ddd05e8b9accbb42e0be3fe79a00036c8ca1",
|
| 64 |
+
"viewer/app.js": "f1fc8b2b559a253b4e0aa228fc1d69cd3724048b442136b10566596ff4c50abf",
|
| 65 |
"viewer/camera-follow.js": "eb7f768c929f5d5933b167a5380cb308c0a5dca7e047cbff1f858025c1021cd5",
|
| 66 |
"viewer/camera-layer.js": "96c625b0769a2ce8b6826556f781e75d536fb1f65f626c61d9752955981361b0",
|
| 67 |
"viewer/controls.js": "1a4f25294e3b2a72c772aeadadfb9d74e1e8772015764cb0a2b49559864a66d6",
|
| 68 |
"viewer/dynamic-tracks.js": "d676672d0a7359a68d2150289c07db432b071b7ac12d8b566c33240fa9176493",
|
| 69 |
+
"viewer/frame-image-cache.js": "25bf260dc5be48d5cd779125489914f3511c28430b1fd87c664069e3be291fbb",
|
| 70 |
+
"viewer/index.html": "0bdbab8ee9c097d7c27067bc99212de5bfb66cce9b4aacfd6578b50f7c1abe6b",
|
| 71 |
"viewer/point-cloud.js": "9a75bd0b78bdbfb7bfd925422e2b748adaa7e43465a84667e7c7da464deea874",
|
| 72 |
"viewer/point-picking.js": "de7bddd93d60bdd249c3df7b68e98637102125581487e711c1ad4d2a8090512a",
|
| 73 |
"viewer/point-sampling.js": "5915c7a7e67afbe9e59082980276accf44f2a3e6e9d9f6da8759bacbb47e4bd6",
|
| 74 |
+
"viewer/rgb-preview.js": "93aefdfffd5b0a31e85dbe5de2b2db133e17de1bff708820298cc628ed67c698",
|
| 75 |
"viewer/serve.py": "4b4b51392a4ee4c43c3178ca5520bba5627d442aacc070ab4cc8c2f5b7d22644",
|
| 76 |
"viewer/sort-worker.js": "03f9041725a46b388dffc7bda85d952d412510bca30b5687d5556c6ab7605670",
|
| 77 |
"viewer/static-demo-data.js": "ba8ddf67ff151168c6d97310458241faafc2f53a4763abe34739e0ca66b7e865",
|
| 78 |
"viewer/static-demo-viewer.js": "3b60da852877c49cbbc277bf5803540927600d3c945644e9c193eab4f295246e",
|
| 79 |
+
"viewer/style.css": "198c37dc10b3f839e32d4f46ed3ee87d172861fc21350e0c493c09e7fb7a8df3",
|
| 80 |
"viewer/vendor/TrackballControls.js": "34e097b83260d8f40d23473bc7511889909660f89de9734e46928df9128fd2a8",
|
| 81 |
"viewer/vendor/three.core.min.js": "61ba0df005b05991361d040d8ff670e1aadfd0ce7aeebd1fdb0725957a8957de",
|
| 82 |
"viewer/vendor/three.module.min.js": "e2b5ee6bccd38fd6d8a2428546b83c5f2426d84b152ef82be8055556e3b40eb6"
|
| 83 |
},
|
| 84 |
+
"updated_at": "2026-11-16T06:15:57.559443+00:00"
|
| 85 |
}
|
space.css
CHANGED
|
@@ -53,9 +53,6 @@ body, .gradio-container { background: #f7f7f9 !important; color: #26272e !import
|
|
| 53 |
.nav-note-right { transform: translateX(-50%) rotate(4deg); }
|
| 54 |
.nav-external { border-color: #26272e2e; background: #fff; box-shadow: 0 4px 14px #272b3808; }
|
| 55 |
.nav-external:hover { border-color: #7c3aed70; color: #6d28d9; box-shadow: 0 7px 20px #7c3aed14; transform: translateY(-1px); }
|
| 56 |
-
.nav-demo { gap: 12px; margin-left: 4px; border-color: var(--ink); background: var(--ink); color: var(--paper); }
|
| 57 |
-
.nav-demo:hover { border-color: #454650; background: #454650; color: #fff; }
|
| 58 |
-
.demo-arrow { transform: rotate(90deg); }
|
| 59 |
.site-header a:focus-visible, .site-footer a:focus-visible { outline: 2px solid #48576b; outline-offset: 4px; }
|
| 60 |
.empty-scene .scene-title { display: flex; align-items: center; justify-content: center; gap: .25em; max-width: 100%; margin: 0; color: #26272e; font: 400 min(30px, 3.55cqw) / 1.3 var(--serif); letter-spacing: -.035em; white-space: nowrap; }
|
| 61 |
.hero-title-brand { display: inline-flex; align-items: baseline; gap: 4px; flex-shrink: 0; color: #7c3aed; }
|
|
@@ -141,8 +138,7 @@ body, .gradio-container { background: #f7f7f9 !important; color: #26272e !import
|
|
| 141 |
@media (max-width: 1280px) {
|
| 142 |
.site-header { flex-wrap: wrap; gap: 16px; padding: 20px 0 36px; }
|
| 143 |
.site-header .brand-lockup { width: 100%; justify-content: space-between; }
|
| 144 |
-
.site-header nav { width: 100%; display: grid; grid-template-columns: repeat(
|
| 145 |
-
.nav-demo { margin-left: 0; }
|
| 146 |
}
|
| 147 |
@media (max-width: 1050px) {
|
| 148 |
#pi4-examples { grid-template-columns: repeat(2, minmax(0, 1fr)); }
|
|
@@ -157,7 +153,7 @@ body, .gradio-container { background: #f7f7f9 !important; color: #26272e !import
|
|
| 157 |
.site-header { min-height: 78px; gap: 16px; }
|
| 158 |
.wordmark { gap: 12px; }
|
| 159 |
.site-header .paper-logo, .site-footer .paper-logo { width: 36px; }
|
| 160 |
-
.site-header nav { grid-template-columns: repeat(
|
| 161 |
.nav-button { min-height: 44px; padding: 0 10px; }
|
| 162 |
.footer-main { flex-wrap: wrap; }
|
| 163 |
#pi4-workspace, #pi4-controls { flex-direction: column; gap: 16px; }
|
|
|
|
| 53 |
.nav-note-right { transform: translateX(-50%) rotate(4deg); }
|
| 54 |
.nav-external { border-color: #26272e2e; background: #fff; box-shadow: 0 4px 14px #272b3808; }
|
| 55 |
.nav-external:hover { border-color: #7c3aed70; color: #6d28d9; box-shadow: 0 7px 20px #7c3aed14; transform: translateY(-1px); }
|
|
|
|
|
|
|
|
|
|
| 56 |
.site-header a:focus-visible, .site-footer a:focus-visible { outline: 2px solid #48576b; outline-offset: 4px; }
|
| 57 |
.empty-scene .scene-title { display: flex; align-items: center; justify-content: center; gap: .25em; max-width: 100%; margin: 0; color: #26272e; font: 400 min(30px, 3.55cqw) / 1.3 var(--serif); letter-spacing: -.035em; white-space: nowrap; }
|
| 58 |
.hero-title-brand { display: inline-flex; align-items: baseline; gap: 4px; flex-shrink: 0; color: #7c3aed; }
|
|
|
|
| 138 |
@media (max-width: 1280px) {
|
| 139 |
.site-header { flex-wrap: wrap; gap: 16px; padding: 20px 0 36px; }
|
| 140 |
.site-header .brand-lockup { width: 100%; justify-content: space-between; }
|
| 141 |
+
.site-header nav { width: 100%; display: grid; grid-template-columns: repeat(4, minmax(0, 1fr)); gap: 10px; font-size: 12px; }
|
|
|
|
| 142 |
}
|
| 143 |
@media (max-width: 1050px) {
|
| 144 |
#pi4-examples { grid-template-columns: repeat(2, minmax(0, 1fr)); }
|
|
|
|
| 153 |
.site-header { min-height: 78px; gap: 16px; }
|
| 154 |
.wordmark { gap: 12px; }
|
| 155 |
.site-header .paper-logo, .site-footer .paper-logo { width: 36px; }
|
| 156 |
+
.site-header nav { grid-template-columns: repeat(2, minmax(0, 1fr)); gap: 32px 10px; font-size: 12px; }
|
| 157 |
.nav-button { min-height: 44px; padding: 0 10px; }
|
| 158 |
.footer-main { flex-wrap: wrap; }
|
| 159 |
#pi4-workspace, #pi4-controls { flex-direction: column; gap: 16px; }
|
viewer/app.js
CHANGED
|
@@ -2,12 +2,13 @@ import * as THREE from 'three';
|
|
| 2 |
import {ScreenTrackballControls} from './controls.js';
|
| 3 |
import {PointCloud} from './point-cloud.js';
|
| 4 |
import {pickPointCloudPoint} from './point-picking.js';
|
| 5 |
-
import {
|
| 6 |
import {StaticDemoViewer} from './static-demo-viewer.js';
|
| 7 |
import {predictedCameraView, SmoothCameraFollower} from './camera-follow.js';
|
| 8 |
import {CameraLayer} from './camera-layer.js';
|
| 9 |
import {DEFAULT_POINT_BUDGET, linspaceIndices} from './point-sampling.js';
|
| 10 |
import {FrameImageCache} from './frame-image-cache.js';
|
|
|
|
| 11 |
|
| 12 |
const byId = id => document.getElementById(id);
|
| 13 |
const formatPoints = count => count >= 1_000_000 ? `${(count / 1_000_000).toFixed(2)}M` : count >= 1_000 ? `${(count / 1_000).toFixed(count % 1_000 ? 1 : 0)}K` : String(count);
|
|
@@ -52,6 +53,7 @@ let seekPromise;
|
|
| 52 |
let budgetTimer;
|
| 53 |
let filmstripImages = [];
|
| 54 |
let frameImages;
|
|
|
|
| 55 |
let lastRender = 0;
|
| 56 |
let lastAnimation = performance.now();
|
| 57 |
let followDistance = 1;
|
|
@@ -67,16 +69,19 @@ function drawFilmstrip() {
|
|
| 67 |
if (!context || !filmstripImages.length) return;
|
| 68 |
const width = canvas.width / filmstripImages.length;
|
| 69 |
filmstripImages.forEach((image, index) => {
|
| 70 |
-
if (image.complete && image.naturalWidth) context.drawImage(image, index * width, 0, width + 1, canvas.height);
|
| 71 |
});
|
| 72 |
}
|
| 73 |
|
| 74 |
function loadFilmstrip(description) {
|
| 75 |
-
|
| 76 |
-
|
| 77 |
-
|
| 78 |
-
|
| 79 |
-
|
|
|
|
|
|
|
|
|
|
| 80 |
});
|
| 81 |
}
|
| 82 |
|
|
@@ -324,7 +329,7 @@ window.addEventListener('pagehide', event => {
|
|
| 324 |
scrubState = null;
|
| 325 |
clearTimeout(budgetTimer);
|
| 326 |
if (event.persisted) demoViewer?.stopPlayback();
|
| 327 |
-
else { demoViewer?.dispose(); frameImages?.dispose(); cloud?.clearSorter(); }
|
| 328 |
});
|
| 329 |
window.addEventListener('keydown', event => {
|
| 330 |
if (['INPUT', 'SELECT', 'TEXTAREA', 'BUTTON'].includes(document.activeElement?.tagName) || event.metaKey || event.ctrlKey || event.altKey) return;
|
|
@@ -350,7 +355,6 @@ async function open() {
|
|
| 350 |
const description = await readJSON('scene.json');
|
| 351 |
if (description.format !== 'pi4_scene_v1' || !description.viewer || !description.frames.length) throw new Error('Run example.py with --viewer first.');
|
| 352 |
const source = description.source ?? {};
|
| 353 |
-
byId('scene-meta').textContent = [`${description.frames.length} frames`, source.image_size?.join(' × ')].filter(Boolean).join(' · ');
|
| 354 |
byId('source-name').textContent = String(source.source ?? '').split(/[\\/]/).filter(Boolean).at(-1) || 'RGB sequence';
|
| 355 |
byId('source-name').title = byId('source-name').textContent;
|
| 356 |
const capacity = description.frames.reduce((total, frame) => total + frame.count, 0);
|
|
@@ -366,6 +370,7 @@ async function open() {
|
|
| 366 |
byId('point-budget-max').textContent = formatPoints(Number(budgetSlider.max));
|
| 367 |
updatePointBudget(true);
|
| 368 |
frameImages = new FrameImageCache(description.frames);
|
|
|
|
| 369 |
byId('confidence').disabled = !description.viewer.confidence;
|
| 370 |
byId('confidence').value = description.viewer.confidence?.default_threshold ?? 0;
|
| 371 |
if (byId('confidence').disabled) byId('confidence-note').textContent = 'This older export has no geometry confidence. Re-export with example.py --viewer to enable filtering.';
|
|
@@ -408,12 +413,12 @@ async function open() {
|
|
| 408 |
if (tracks && !tracks.arrays.confidence) byId('track-confidence-note').textContent = 'This export has no tracking confidence; filtering is unavailable.';
|
| 409 |
applySettings();
|
| 410 |
},
|
| 411 |
-
frame:
|
| 412 |
if (!scrubState) byId('frame').value = frame;
|
| 413 |
byId('frame').setAttribute('aria-valuetext', `Frame ${frame + 1} of ${description.frames.length}`);
|
| 414 |
byId('frame-value').textContent = `${frame + 1} / ${description.frames.length}`;
|
| 415 |
updateRangeStyle(byId('frame'));
|
| 416 |
-
frameImages.show(frame, byId('preview'));
|
| 417 |
updateFollowCamera();
|
| 418 |
updateStatus();
|
| 419 |
updateClipping();
|
|
|
|
| 2 |
import {ScreenTrackballControls} from './controls.js';
|
| 3 |
import {PointCloud} from './point-cloud.js';
|
| 4 |
import {pickPointCloudPoint} from './point-picking.js';
|
| 5 |
+
import {readJSON} from './static-demo-data.js';
|
| 6 |
import {StaticDemoViewer} from './static-demo-viewer.js';
|
| 7 |
import {predictedCameraView, SmoothCameraFollower} from './camera-follow.js';
|
| 8 |
import {CameraLayer} from './camera-layer.js';
|
| 9 |
import {DEFAULT_POINT_BUDGET, linspaceIndices} from './point-sampling.js';
|
| 10 |
import {FrameImageCache} from './frame-image-cache.js';
|
| 11 |
+
import {VideoFramePreview} from './rgb-preview.js';
|
| 12 |
|
| 13 |
const byId = id => document.getElementById(id);
|
| 14 |
const formatPoints = count => count >= 1_000_000 ? `${(count / 1_000_000).toFixed(2)}M` : count >= 1_000 ? `${(count / 1_000).toFixed(count % 1_000 ? 1 : 0)}K` : String(count);
|
|
|
|
| 53 |
let budgetTimer;
|
| 54 |
let filmstripImages = [];
|
| 55 |
let frameImages;
|
| 56 |
+
let videoPreview;
|
| 57 |
let lastRender = 0;
|
| 58 |
let lastAnimation = performance.now();
|
| 59 |
let followDistance = 1;
|
|
|
|
| 69 |
if (!context || !filmstripImages.length) return;
|
| 70 |
const width = canvas.width / filmstripImages.length;
|
| 71 |
filmstripImages.forEach((image, index) => {
|
| 72 |
+
if (image?.complete && image.naturalWidth) context.drawImage(image, index * width, 0, width + 1, canvas.height);
|
| 73 |
});
|
| 74 |
}
|
| 75 |
|
| 76 |
function loadFilmstrip(description) {
|
| 77 |
+
const frames = Array.from(linspaceIndices(description.frames.length, 12));
|
| 78 |
+
filmstripImages = Array(frames.length).fill(null);
|
| 79 |
+
frames.forEach((frame, index) => {
|
| 80 |
+
frameImages.getImage(frame).then(image => {
|
| 81 |
+
if (!image) return;
|
| 82 |
+
filmstripImages[index] = image;
|
| 83 |
+
drawFilmstrip();
|
| 84 |
+
});
|
| 85 |
});
|
| 86 |
}
|
| 87 |
|
|
|
|
| 329 |
scrubState = null;
|
| 330 |
clearTimeout(budgetTimer);
|
| 331 |
if (event.persisted) demoViewer?.stopPlayback();
|
| 332 |
+
else { demoViewer?.dispose(); videoPreview?.dispose(); frameImages?.dispose(); cloud?.clearSorter(); }
|
| 333 |
});
|
| 334 |
window.addEventListener('keydown', event => {
|
| 335 |
if (['INPUT', 'SELECT', 'TEXTAREA', 'BUTTON'].includes(document.activeElement?.tagName) || event.metaKey || event.ctrlKey || event.altKey) return;
|
|
|
|
| 355 |
const description = await readJSON('scene.json');
|
| 356 |
if (description.format !== 'pi4_scene_v1' || !description.viewer || !description.frames.length) throw new Error('Run example.py with --viewer first.');
|
| 357 |
const source = description.source ?? {};
|
|
|
|
| 358 |
byId('source-name').textContent = String(source.source ?? '').split(/[\\/]/).filter(Boolean).at(-1) || 'RGB sequence';
|
| 359 |
byId('source-name').title = byId('source-name').textContent;
|
| 360 |
const capacity = description.frames.reduce((total, frame) => total + frame.count, 0);
|
|
|
|
| 370 |
byId('point-budget-max').textContent = formatPoints(Number(budgetSlider.max));
|
| 371 |
updatePointBudget(true);
|
| 372 |
frameImages = new FrameImageCache(description.frames);
|
| 373 |
+
videoPreview = VideoFramePreview.fromInput(source, (frame, canvas) => frameImages.show(frame, canvas));
|
| 374 |
byId('confidence').disabled = !description.viewer.confidence;
|
| 375 |
byId('confidence').value = description.viewer.confidence?.default_threshold ?? 0;
|
| 376 |
if (byId('confidence').disabled) byId('confidence-note').textContent = 'This older export has no geometry confidence. Re-export with example.py --viewer to enable filtering.';
|
|
|
|
| 413 |
if (tracks && !tracks.arrays.confidence) byId('track-confidence-note').textContent = 'This export has no tracking confidence; filtering is unavailable.';
|
| 414 |
applySettings();
|
| 415 |
},
|
| 416 |
+
frame: frame => {
|
| 417 |
if (!scrubState) byId('frame').value = frame;
|
| 418 |
byId('frame').setAttribute('aria-valuetext', `Frame ${frame + 1} of ${description.frames.length}`);
|
| 419 |
byId('frame-value').textContent = `${frame + 1} / ${description.frames.length}`;
|
| 420 |
updateRangeStyle(byId('frame'));
|
| 421 |
+
if (!videoPreview?.show(frame, byId('preview'))) frameImages.show(frame, byId('preview'));
|
| 422 |
updateFollowCamera();
|
| 423 |
updateStatus();
|
| 424 |
updateClipping();
|
viewer/frame-image-cache.js
CHANGED
|
@@ -1,56 +1,69 @@
|
|
| 1 |
import {dataURL} from './static-demo-data.js';
|
|
|
|
| 2 |
|
| 3 |
export class FrameImageCache {
|
| 4 |
constructor(frames) {
|
| 5 |
this.frames = frames;
|
| 6 |
this.entries = new Map();
|
| 7 |
-
this.
|
| 8 |
this.current = 0;
|
| 9 |
this.ahead = 24;
|
| 10 |
this.behind = 4;
|
| 11 |
this.active = 0;
|
| 12 |
this.concurrency = 3;
|
|
|
|
| 13 |
this.generation = 0;
|
| 14 |
this.disposed = false;
|
|
|
|
| 15 |
}
|
| 16 |
|
| 17 |
-
show(frame,
|
| 18 |
if (this.disposed) return;
|
| 19 |
this.current = frame;
|
| 20 |
const generation = ++this.generation;
|
|
|
|
|
|
|
|
|
|
| 21 |
const first = Math.max(0, frame - this.behind);
|
| 22 |
const last = Math.min(this.frames.length - 1, frame + this.ahead);
|
| 23 |
-
for (
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 28 |
}
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
const entry = {frame: index, started: false, ready: false};
|
| 32 |
entry.promise = new Promise(resolve => { entry.resolve = resolve; });
|
| 33 |
-
|
| 34 |
}
|
| 35 |
-
const
|
| 36 |
-
const display = ready => {
|
| 37 |
-
if (this.disposed || generation !== this.generation) return;
|
| 38 |
-
const image = entry.image;
|
| 39 |
-
if (image === element) return;
|
| 40 |
-
image.id = element.id;
|
| 41 |
-
image.alt = ready ? 'Input RGB frame' : `RGB frame ${frame + 1} unavailable`;
|
| 42 |
-
element.removeAttribute('id');
|
| 43 |
-
element.replaceWith(image);
|
| 44 |
-
};
|
| 45 |
-
if (entry.ready) display(true);
|
| 46 |
-
else entry.promise.then(display);
|
| 47 |
this.pump();
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 48 |
}
|
| 49 |
|
| 50 |
pump() {
|
| 51 |
if (this.disposed) return;
|
| 52 |
const priority = entry => entry.frame >= this.current ? entry.frame - this.current : this.frames.length + this.current - entry.frame;
|
| 53 |
-
const queued = [...this.entries.values()].filter(entry => !entry.started).sort((left, right) => priority(left) - priority(right));
|
| 54 |
for (const entry of queued) {
|
| 55 |
if (this.active >= this.concurrency) break;
|
| 56 |
this.load(entry);
|
|
@@ -59,20 +72,43 @@ export class FrameImageCache {
|
|
| 59 |
|
| 60 |
async load(entry) {
|
| 61 |
entry.started = true;
|
|
|
|
| 62 |
this.active++;
|
| 63 |
-
|
| 64 |
-
entry.image = new Image();
|
| 65 |
-
entry.image.decoding = 'async';
|
| 66 |
try {
|
| 67 |
-
|
| 68 |
-
|
| 69 |
-
|
| 70 |
-
|
| 71 |
-
|
| 72 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 73 |
} finally {
|
|
|
|
|
|
|
|
|
|
|
|
|
| 74 |
this.active--;
|
| 75 |
-
this.inflight.delete(entry);
|
| 76 |
this.pump();
|
| 77 |
}
|
| 78 |
}
|
|
@@ -80,10 +116,12 @@ export class FrameImageCache {
|
|
| 80 |
dispose() {
|
| 81 |
this.disposed = true;
|
| 82 |
this.generation++;
|
|
|
|
| 83 |
for (const entry of this.entries.values()) {
|
| 84 |
-
entry.resolve(
|
|
|
|
| 85 |
}
|
| 86 |
-
|
| 87 |
this.entries.clear();
|
| 88 |
}
|
| 89 |
}
|
|
|
|
| 1 |
import {dataURL} from './static-demo-data.js';
|
| 2 |
+
import {drawPreviewFrame} from './rgb-preview.js';
|
| 3 |
|
| 4 |
export class FrameImageCache {
|
| 5 |
constructor(frames) {
|
| 6 |
this.frames = frames;
|
| 7 |
this.entries = new Map();
|
| 8 |
+
this.decoded = new Map();
|
| 9 |
this.current = 0;
|
| 10 |
this.ahead = 24;
|
| 11 |
this.behind = 4;
|
| 12 |
this.active = 0;
|
| 13 |
this.concurrency = 3;
|
| 14 |
+
this.maxDecoded = 32;
|
| 15 |
this.generation = 0;
|
| 16 |
this.disposed = false;
|
| 17 |
+
this.abort = new AbortController();
|
| 18 |
}
|
| 19 |
|
| 20 |
+
show(frame, canvas) {
|
| 21 |
if (this.disposed) return;
|
| 22 |
this.current = frame;
|
| 23 |
const generation = ++this.generation;
|
| 24 |
+
this.getImage(frame).then(image => {
|
| 25 |
+
if (image && !this.disposed && generation === this.generation) drawPreviewFrame(canvas, image, frame, 'image');
|
| 26 |
+
});
|
| 27 |
const first = Math.max(0, frame - this.behind);
|
| 28 |
const last = Math.min(this.frames.length - 1, frame + this.ahead);
|
| 29 |
+
for (let index = first; index <= last; index++) this.getImage(index);
|
| 30 |
+
}
|
| 31 |
+
|
| 32 |
+
getImage(frame) {
|
| 33 |
+
if (this.disposed || !Number.isInteger(frame) || !this.frames[frame]) return Promise.resolve(null);
|
| 34 |
+
let entry = this.entries.get(frame);
|
| 35 |
+
if (!entry) {
|
| 36 |
+
entry = {frame, image: null, url: null, started: false};
|
| 37 |
+
this.entries.set(frame, entry);
|
| 38 |
+
}
|
| 39 |
+
if (entry.image) {
|
| 40 |
+
this.touch(entry);
|
| 41 |
+
return Promise.resolve(entry.image);
|
| 42 |
}
|
| 43 |
+
if (entry.retryAfter > performance.now()) return Promise.resolve(null);
|
| 44 |
+
if (!entry.promise) {
|
|
|
|
| 45 |
entry.promise = new Promise(resolve => { entry.resolve = resolve; });
|
| 46 |
+
entry.queued = true;
|
| 47 |
}
|
| 48 |
+
const pending = entry.promise;
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 49 |
this.pump();
|
| 50 |
+
return pending;
|
| 51 |
+
}
|
| 52 |
+
|
| 53 |
+
touch(entry) {
|
| 54 |
+
this.decoded.delete(entry.frame);
|
| 55 |
+
this.decoded.set(entry.frame, entry);
|
| 56 |
+
while (this.decoded.size > this.maxDecoded) {
|
| 57 |
+
const [frame, oldest] = this.decoded.entries().next().value;
|
| 58 |
+
oldest.image = null;
|
| 59 |
+
this.decoded.delete(frame);
|
| 60 |
+
}
|
| 61 |
}
|
| 62 |
|
| 63 |
pump() {
|
| 64 |
if (this.disposed) return;
|
| 65 |
const priority = entry => entry.frame >= this.current ? entry.frame - this.current : this.frames.length + this.current - entry.frame;
|
| 66 |
+
const queued = [...this.entries.values()].filter(entry => entry.queued && !entry.started).sort((left, right) => priority(left) - priority(right));
|
| 67 |
for (const entry of queued) {
|
| 68 |
if (this.active >= this.concurrency) break;
|
| 69 |
this.load(entry);
|
|
|
|
| 72 |
|
| 73 |
async load(entry) {
|
| 74 |
entry.started = true;
|
| 75 |
+
entry.queued = false;
|
| 76 |
this.active++;
|
| 77 |
+
let image = null;
|
|
|
|
|
|
|
| 78 |
try {
|
| 79 |
+
for (let attempt = 1; attempt <= 3 && !this.disposed; attempt++) {
|
| 80 |
+
let url = entry.url;
|
| 81 |
+
try {
|
| 82 |
+
if (!url) {
|
| 83 |
+
const response = await fetch(dataURL(this.frames[entry.frame].image), {signal: this.abort.signal});
|
| 84 |
+
if (!response.ok) throw new Error(`RGB frame: HTTP ${response.status}`);
|
| 85 |
+
const blob = await response.blob();
|
| 86 |
+
if (this.disposed) break;
|
| 87 |
+
url = URL.createObjectURL(blob);
|
| 88 |
+
}
|
| 89 |
+
const decoded = new Image();
|
| 90 |
+
decoded.decoding = 'async';
|
| 91 |
+
decoded.src = url;
|
| 92 |
+
await decoded.decode();
|
| 93 |
+
if (this.disposed) {
|
| 94 |
+
if (!entry.url) URL.revokeObjectURL(url);
|
| 95 |
+
break;
|
| 96 |
+
}
|
| 97 |
+
entry.url = url;
|
| 98 |
+
entry.image = image = decoded;
|
| 99 |
+
this.touch(entry);
|
| 100 |
+
break;
|
| 101 |
+
} catch {
|
| 102 |
+
if (url && !entry.url) URL.revokeObjectURL(url);
|
| 103 |
+
if (attempt < 3 && !this.disposed) await new Promise(resolve => setTimeout(resolve, attempt * 200));
|
| 104 |
+
}
|
| 105 |
+
}
|
| 106 |
} finally {
|
| 107 |
+
entry.retryAfter = image ? 0 : performance.now() + 5000;
|
| 108 |
+
entry.resolve(image);
|
| 109 |
+
entry.promise = null;
|
| 110 |
+
entry.started = false;
|
| 111 |
this.active--;
|
|
|
|
| 112 |
this.pump();
|
| 113 |
}
|
| 114 |
}
|
|
|
|
| 116 |
dispose() {
|
| 117 |
this.disposed = true;
|
| 118 |
this.generation++;
|
| 119 |
+
this.abort.abort();
|
| 120 |
for (const entry of this.entries.values()) {
|
| 121 |
+
entry.resolve?.(null);
|
| 122 |
+
if (entry.url) URL.revokeObjectURL(entry.url);
|
| 123 |
}
|
| 124 |
+
this.decoded.clear();
|
| 125 |
this.entries.clear();
|
| 126 |
}
|
| 127 |
}
|
viewer/index.html
CHANGED
|
@@ -9,10 +9,6 @@
|
|
| 9 |
<script type="importmap">{"imports":{"three":"./vendor/three.module.min.js"}}</script>
|
| 10 |
</head>
|
| 11 |
<body>
|
| 12 |
-
<header class="topbar">
|
| 13 |
-
<div class="brand"><strong>π<sup>4</sup><span class="brand-dot">.</span></strong><span class="brand-divider"></span><span class="brand-caption">GEOMETRY IN MOTION</span></div>
|
| 14 |
-
<div class="session-meta"><span class="session-dot"></span><span id="scene-meta">Preparing scene</span><span class="session-label">LIVE RECONSTRUCTION</span></div>
|
| 15 |
-
</header>
|
| 16 |
<main id="viewer-shell">
|
| 17 |
<section id="viewport" aria-label="Interactive 3D reconstruction">
|
| 18 |
<div class="scene-heading"><span class="eyebrow">EXPLORE THE FOURTH DIMENSION</span><h1>One scene. Every perspective.</h1><p id="source-name">RGB sequence</p></div>
|
|
@@ -26,7 +22,7 @@
|
|
| 26 |
</div>
|
| 27 |
<button id="enter-camera" class="preview-wrap" title="Enter the current input camera" disabled>
|
| 28 |
<span class="preview-heading"><span class="session-dot"></span>INPUT RGB <span>ENTER CAMERA ↗</span></span>
|
| 29 |
-
<
|
| 30 |
</button>
|
| 31 |
<p id="loading" class="loading-badge" role="status">Preparing scene…</p>
|
| 32 |
<div class="scene-readouts"><div><span class="readout-label">VISIBLE POINTS</span><strong id="visible-points">—</strong></div><div><span class="readout-label">FRAMES LOADED</span><strong id="loaded-frames">—</strong><span class="load-meter"><span id="load-progress"></span></span></div></div>
|
|
|
|
| 9 |
<script type="importmap">{"imports":{"three":"./vendor/three.module.min.js"}}</script>
|
| 10 |
</head>
|
| 11 |
<body>
|
|
|
|
|
|
|
|
|
|
|
|
|
| 12 |
<main id="viewer-shell">
|
| 13 |
<section id="viewport" aria-label="Interactive 3D reconstruction">
|
| 14 |
<div class="scene-heading"><span class="eyebrow">EXPLORE THE FOURTH DIMENSION</span><h1>One scene. Every perspective.</h1><p id="source-name">RGB sequence</p></div>
|
|
|
|
| 22 |
</div>
|
| 23 |
<button id="enter-camera" class="preview-wrap" title="Enter the current input camera" disabled>
|
| 24 |
<span class="preview-heading"><span class="session-dot"></span>INPUT RGB <span>ENTER CAMERA ↗</span></span>
|
| 25 |
+
<canvas id="preview" width="512" height="288" role="img" aria-label="Input RGB frame"></canvas><span id="source-time" class="preview-time">00:00.00</span>
|
| 26 |
</button>
|
| 27 |
<p id="loading" class="loading-badge" role="status">Preparing scene…</p>
|
| 28 |
<div class="scene-readouts"><div><span class="readout-label">VISIBLE POINTS</span><strong id="visible-points">—</strong></div><div><span class="readout-label">FRAMES LOADED</span><strong id="loaded-frames">—</strong><span class="load-meter"><span id="load-progress"></span></span></div></div>
|
viewer/rgb-preview.js
ADDED
|
@@ -0,0 +1,101 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
export function drawPreviewFrame(canvas, image, frame, source, dimensions) {
|
| 2 |
+
const nativeWidth = image.videoWidth || image.naturalWidth;
|
| 3 |
+
const nativeHeight = image.videoHeight || image.naturalHeight;
|
| 4 |
+
if (!nativeWidth || !nativeHeight) return;
|
| 5 |
+
const scale = Math.min(1, 512 / Math.max(nativeWidth, nativeHeight));
|
| 6 |
+
const validSize = Array.isArray(dimensions) && dimensions.length === 2 && dimensions.every(size => Number.isSafeInteger(size) && size > 0 && size <= 512);
|
| 7 |
+
const width = validSize ? dimensions[0] : Math.max(1, Math.round(nativeWidth * scale));
|
| 8 |
+
const height = validSize ? dimensions[1] : Math.max(1, Math.round(nativeHeight * scale));
|
| 9 |
+
if (canvas.width !== width || canvas.height !== height) {
|
| 10 |
+
canvas.width = width;
|
| 11 |
+
canvas.height = height;
|
| 12 |
+
}
|
| 13 |
+
canvas.getContext('2d').drawImage(image, 0, 0, width, height);
|
| 14 |
+
canvas.dataset.frame = frame;
|
| 15 |
+
canvas.dataset.source = source;
|
| 16 |
+
}
|
| 17 |
+
|
| 18 |
+
export class VideoFramePreview {
|
| 19 |
+
static fromInput(source, fallback) {
|
| 20 |
+
const timestamps = source.source_timestamps_s;
|
| 21 |
+
if (!Array.isArray(timestamps) || !timestamps.length || !timestamps.every(time => Number.isFinite(time) && time >= 0)) return null;
|
| 22 |
+
try {
|
| 23 |
+
const expected = window.frameElement?.getAttribute('data-source-video');
|
| 24 |
+
const input = window.parent.document.querySelector('#pi4-video video');
|
| 25 |
+
if (!expected || !input?.currentSrc) return null;
|
| 26 |
+
const expectedURL = new URL(expected, window.location.href);
|
| 27 |
+
const inputURL = new URL(input.currentSrc, window.location.href);
|
| 28 |
+
if (expectedURL.origin !== inputURL.origin || decodeURIComponent(expectedURL.pathname) !== decodeURIComponent(inputURL.pathname)) return null;
|
| 29 |
+
return new VideoFramePreview(input, timestamps, fallback, source.image_size);
|
| 30 |
+
} catch {
|
| 31 |
+
return null;
|
| 32 |
+
}
|
| 33 |
+
}
|
| 34 |
+
|
| 35 |
+
constructor(input, timestamps, fallback, dimensions) {
|
| 36 |
+
this.input = input;
|
| 37 |
+
this.sourceURL = input.currentSrc;
|
| 38 |
+
this.timestamps = timestamps;
|
| 39 |
+
this.fallback = fallback;
|
| 40 |
+
this.dimensions = dimensions;
|
| 41 |
+
this.failed = false;
|
| 42 |
+
this.disposed = false;
|
| 43 |
+
this.video = document.createElement('video');
|
| 44 |
+
this.video.muted = true;
|
| 45 |
+
this.video.playsInline = true;
|
| 46 |
+
this.video.preload = 'auto';
|
| 47 |
+
this.update = () => this.seek();
|
| 48 |
+
this.fail = () => {
|
| 49 |
+
if (this.disposed || this.failed) return;
|
| 50 |
+
this.failed = true;
|
| 51 |
+
clearTimeout(this.timeout);
|
| 52 |
+
this.release();
|
| 53 |
+
if (this.requested) this.fallback(this.requested.frame, this.requested.canvas);
|
| 54 |
+
};
|
| 55 |
+
for (const event of ['loadedmetadata', 'loadeddata', 'seeked', 'canplay']) this.video.addEventListener(event, this.update);
|
| 56 |
+
this.video.addEventListener('error', this.fail);
|
| 57 |
+
this.video.src = this.sourceURL;
|
| 58 |
+
this.video.load();
|
| 59 |
+
}
|
| 60 |
+
|
| 61 |
+
show(frame, canvas) {
|
| 62 |
+
if (this.disposed || this.failed || !Number.isFinite(this.timestamps[frame])) return false;
|
| 63 |
+
this.requested = {frame, canvas, time: this.timestamps[frame]};
|
| 64 |
+
if (!this.timeout) this.timeout = setTimeout(this.fail, 12000);
|
| 65 |
+
this.seek();
|
| 66 |
+
return !this.failed;
|
| 67 |
+
}
|
| 68 |
+
|
| 69 |
+
seek() {
|
| 70 |
+
if (!this.requested || this.disposed || this.failed) return;
|
| 71 |
+
const {frame, canvas, time} = this.requested;
|
| 72 |
+
const media = this.input.currentSrc === this.sourceURL && !this.input.seeking && this.input.readyState >= 2 && Math.abs(this.input.currentTime - time) < .0001 ? this.input : this.video;
|
| 73 |
+
try {
|
| 74 |
+
if (media.readyState < 1 || media.seeking) return;
|
| 75 |
+
if (Number.isFinite(media.duration) && time >= media.duration) return this.fail();
|
| 76 |
+
if (Math.abs(media.currentTime - time) >= .0001) {
|
| 77 |
+
media.currentTime = time;
|
| 78 |
+
return;
|
| 79 |
+
}
|
| 80 |
+
if (media.readyState < 2) return;
|
| 81 |
+
drawPreviewFrame(canvas, media, frame, 'video', this.dimensions);
|
| 82 |
+
clearTimeout(this.timeout);
|
| 83 |
+
this.timeout = null;
|
| 84 |
+
} catch {
|
| 85 |
+
this.fail();
|
| 86 |
+
}
|
| 87 |
+
}
|
| 88 |
+
|
| 89 |
+
release() {
|
| 90 |
+
for (const event of ['loadedmetadata', 'loadeddata', 'seeked', 'canplay']) this.video.removeEventListener(event, this.update);
|
| 91 |
+
this.video.removeEventListener('error', this.fail);
|
| 92 |
+
this.video.removeAttribute('src');
|
| 93 |
+
this.video.load();
|
| 94 |
+
}
|
| 95 |
+
|
| 96 |
+
dispose() {
|
| 97 |
+
this.disposed = true;
|
| 98 |
+
clearTimeout(this.timeout);
|
| 99 |
+
this.release();
|
| 100 |
+
}
|
| 101 |
+
}
|
viewer/style.css
CHANGED
|
@@ -16,17 +16,8 @@ button:hover { background: #f3effc; border-color: #7c3aed44; }
|
|
| 16 |
button:active { transform: translateY(1px); }
|
| 17 |
button:disabled { cursor: default; opacity: .45; }
|
| 18 |
svg { width: 17px; height: 17px; stroke: currentColor; stroke-width: 1.5; stroke-linecap: round; stroke-linejoin: round; }
|
| 19 |
-
.topbar { height: 70px; padding: 0 30px; display: flex; align-items: center; justify-content: space-between; gap: 20px; }
|
| 20 |
-
.brand { display: flex; align-items: center; gap: 16px; }
|
| 21 |
-
.brand strong { font-size: 35px; font-weight: 650; letter-spacing: -2px; line-height: 1; }
|
| 22 |
-
.brand sup { font-size: 17px; position: relative; top: -3px; margin-left: 2px; }
|
| 23 |
-
.brand-dot { color: var(--accent); }
|
| 24 |
-
.brand-divider { height: 19px; width: 1px; background: #28213923; }
|
| 25 |
-
.brand-caption { font: 9px var(--mono); letter-spacing: 1.6px; color: #757080; }
|
| 26 |
-
.session-meta { display: flex; align-items: center; gap: 9px; font: 10px var(--mono); color: #6e687a; }
|
| 27 |
.session-dot { display: inline-block; width: 5px; height: 5px; border-radius: 50%; flex-shrink: 0; background: var(--accent); box-shadow: 0 0 0 4px #7c3aed0c; }
|
| 28 |
-
|
| 29 |
-
main { height: calc(100dvh - 70px); display: grid; grid-template-columns: minmax(0, 1fr) 290px; gap: 16px; padding: 0 20px 20px; }
|
| 30 |
#viewport { position: relative; isolation: isolate; min-width: 0; min-height: 0; overflow: hidden; border: 1px solid #28213916; border-radius: 16px; background: radial-gradient(ellipse at 40% 45%, #fff 15%, #f7f6fb 80%); box-shadow: 0 16px 50px #2a19460a; }
|
| 31 |
#viewport > canvas { position: absolute; inset: 0; display: block; width: 100%; height: 100%; touch-action: none; }
|
| 32 |
.scene-heading { position: absolute; top: 27px; left: 27px; pointer-events: none; }
|
|
@@ -132,7 +123,7 @@ select { display: block; width: 100%; padding: 10px; border: 1px solid #28213916
|
|
| 132 |
:focus-visible { outline: 2px solid #a278db; outline-offset: 4px; }
|
| 133 |
#viewer-shell:fullscreen { height: 100dvh; padding: 16px; background: #f6f5fa; }
|
| 134 |
@media (max-width: 1100px) {
|
| 135 |
-
main { grid-template-columns: minmax(0, 1fr) 260px; gap: 12px; padding:
|
| 136 |
.camera-mode { display: none; }
|
| 137 |
.scene-heading { top: 23px; left: 21px; }
|
| 138 |
h1 { max-width: 230px; font-size: 18px; }
|
|
@@ -144,11 +135,7 @@ select { display: block; width: 100%; padding: 10px; border: 1px solid #28213916
|
|
| 144 |
}
|
| 145 |
@media (max-width: 720px) {
|
| 146 |
body { overflow: auto; }
|
| 147 |
-
|
| 148 |
-
.brand-caption, .brand-divider, .session-label { display: none; }
|
| 149 |
-
.brand strong { font-size: 30px; }
|
| 150 |
-
.session-meta { font-size: 9px; }
|
| 151 |
-
main { height: auto; grid-template-columns: minmax(0, 1fr); gap: 12px; padding: 0 10px 10px; }
|
| 152 |
#viewport { height: 76dvh; min-height: 470px; }
|
| 153 |
.scene-heading { top: 20px; left: 18px; }
|
| 154 |
.scene-heading .eyebrow { font-size: 6px; letter-spacing: 1px; }
|
|
|
|
| 16 |
button:active { transform: translateY(1px); }
|
| 17 |
button:disabled { cursor: default; opacity: .45; }
|
| 18 |
svg { width: 17px; height: 17px; stroke: currentColor; stroke-width: 1.5; stroke-linecap: round; stroke-linejoin: round; }
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
.session-dot { display: inline-block; width: 5px; height: 5px; border-radius: 50%; flex-shrink: 0; background: var(--accent); box-shadow: 0 0 0 4px #7c3aed0c; }
|
| 20 |
+
main { height: 100dvh; display: grid; grid-template-columns: minmax(0, 1fr) 290px; gap: 16px; padding: 12px 20px 20px; }
|
|
|
|
| 21 |
#viewport { position: relative; isolation: isolate; min-width: 0; min-height: 0; overflow: hidden; border: 1px solid #28213916; border-radius: 16px; background: radial-gradient(ellipse at 40% 45%, #fff 15%, #f7f6fb 80%); box-shadow: 0 16px 50px #2a19460a; }
|
| 22 |
#viewport > canvas { position: absolute; inset: 0; display: block; width: 100%; height: 100%; touch-action: none; }
|
| 23 |
.scene-heading { position: absolute; top: 27px; left: 27px; pointer-events: none; }
|
|
|
|
| 123 |
:focus-visible { outline: 2px solid #a278db; outline-offset: 4px; }
|
| 124 |
#viewer-shell:fullscreen { height: 100dvh; padding: 16px; background: #f6f5fa; }
|
| 125 |
@media (max-width: 1100px) {
|
| 126 |
+
main { grid-template-columns: minmax(0, 1fr) 260px; gap: 12px; padding: 12px; }
|
| 127 |
.camera-mode { display: none; }
|
| 128 |
.scene-heading { top: 23px; left: 21px; }
|
| 129 |
h1 { max-width: 230px; font-size: 18px; }
|
|
|
|
| 135 |
}
|
| 136 |
@media (max-width: 720px) {
|
| 137 |
body { overflow: auto; }
|
| 138 |
+
main { height: auto; grid-template-columns: minmax(0, 1fr); gap: 12px; padding: 10px; }
|
|
|
|
|
|
|
|
|
|
|
|
|
| 139 |
#viewport { height: 76dvh; min-height: 470px; }
|
| 140 |
.scene-heading { top: 20px; left: 18px; }
|
| 141 |
.scene-heading .eyebrow { font-size: 6px; letter-spacing: 1px; }
|