quyu666 commited on
Commit
edae745
·
1 Parent(s): 2afbe59

sync processor from GitLab 2bbf9fe: UMI SLAM 动作模块、批次指纹与设备进度、导出/路由/前端更新

Browse files
processor/README.md CHANGED
@@ -1,3 +1,5 @@
 
 
1
  # processor
2
 
3
  [English](README.md) | [中文](README.zh-CN.md)
@@ -159,12 +161,12 @@ the current Episode.
159
 
160
  ### Video sources
161
 
162
- | Type | Example source name | Description |
163
- | --- | --- | --- |
164
- | Mono RGB | `head_rgb` | Regular RGB video |
165
- | Metric depth | `head_depth` | Independent metric-depth video |
166
- | Stereo RGB | `stereo_left_rgb`, `stereo_right_rgb` | Left and right RGB videos |
167
- | Stereo depth | `stereo_depth` | Independent depth video |
168
 
169
  RGB and Depth are always stored as independent streams. A depth-only stream is never treated as
170
  RGB and is never duplicated alongside the RGB stream.
@@ -250,24 +252,24 @@ when old workflows are loaded; they are not shown in the new-workflow palette.
250
 
251
  #### Capture modules (Input)
252
 
253
- | Frontend module | Inputs | Outputs | Purpose and rules |
254
- | --- | --- | --- | --- |
255
- | `RGB Camera` | None | `RGB Video` | Mono RGB video. The device is selected from the current project's data; blank means auto-match. |
256
- | `RGB-D Camera` | None | `RGB Video`, `Depth` | Emits RGB and real depth streams from the same device; Depth can connect only to a depth input. |
257
- | `Stereo RGB Camera` | None | `Left RGB Video`, `Right RGB Video` | Stereo RGB input. Connecting one side creates the corresponding left/right relationship. |
258
- | `Stereo RGB-D Camera` | None | `Left RGB Video`, `Right RGB Video`, `Depth` | Stereo RGB plus real depth; Depth is used by RGB-D 3D processing. |
259
- | `Glove Sensor` | None | `Glove Sensor Data` | Pressure, joint, or glove sensor data; no video output. It can connect directly to quality review. |
260
 
261
  #### Video and keypoint processing modules (Process)
262
 
263
- | Frontend module | Inputs | Outputs | Purpose and rules |
264
- | --- | --- | --- | --- |
265
- | `Human Annotation` | `RGB Video` | `Annotation` | Frame-level manual annotation on RGB video. |
266
- | `AI Annotation` | `RGB Video` | `Annotation` | Video/segment annotation through a local or API VLM; API is recommended for deployment. |
267
- | `RGB_TO_2D_BareHand` | `RGB Video` | `Hand 2D` | Bare-hand 2D keypoints. `Spatial` is display-only and exports no metric 3D. |
268
- | `RGB_TO_2D_BlackGlove` | `RGB Video` | `Hand 2D` | Black-glove 2D keypoints. Without Depth, it does not calculate metric 3D. |
269
- | `RGB-D_3D_BareHand` | `RGB Video`, `Depth` | `Hand 3D` | Metric 3D bare-hand keypoints from RGB and real Depth; a missing branch is skipped. |
270
- | `RGB-D_3D_BlackGlove` | `RGB Video`, `Depth` | `Hand 3D` | Metric 3D black-glove keypoints from RGB and real Depth; it does not search the disk or fall back to RGB estimation. |
271
 
272
  `MediaPipe Hand` remains available for historical workflows and backend compatibility but is hidden
273
  from the new-workflow palette. All nodes provide hover text describing their purpose and accepted
@@ -275,32 +277,32 @@ connection types.
275
 
276
  #### Review modules (Review)
277
 
278
- | Frontend module | Inputs | Outputs | Purpose and rules |
279
- | --- | --- | --- | --- |
280
- | `Human Review` | `Review Target` | `Reviewed Data` | Human inspection of video, keypoints, or annotation results before downstream processing. |
281
  | `AI Quality Review` | `Quality Review Target` | `Reviewed Data` | Checks video decoding, frame continuity, black screens, freezes, annotation coverage, and sensor data. |
282
 
283
  #### Export modules (Export)
284
 
285
- | Frontend module | Inputs | Outputs | Purpose and rules |
286
- | --- | --- | --- | --- |
287
  | `LeRobot Export` | `Exportable Data` | `Dataset` | Exports LeRobot v2.1 or v3.0; the version is selected on the node. |
288
- | `HDF5 Export` | `Exportable Data` | `Dataset` | Exports HDF5 with the configured compression parameters. |
289
 
290
  #### Workflow Studio UI components
291
 
292
  <details>
293
  <summary>Click to expand: frontend component reference</summary>
294
 
295
- | UI component | Function |
296
- | --- | --- |
297
- | `NodePalette` | Displays Input, Process, Review, and Export categories; supports search, category collapse, and hover descriptions. |
298
- | `WorkflowCanvas` | Supports drag-and-drop node creation, typed-port connections, node movement, selection, zoom, grid snapping, and minimap navigation. |
299
- | `WorkflowNode` | Renders the card title, ports, project device selector/input, `Spatial` button, and API settings button. |
300
- | `WorkflowDrawer` | Lists workflows, creates new workflows, loads workflows, and refreshes project-specific input sources. |
301
- | `PipelineToolbar` | Provides New, Save, Save As, workflow JSON Export, and Run actions; the yellow dot indicates unsaved changes. |
302
- | `NodeSettingsModal` | Configures AI Annotation API vendor, model, endpoint, and key fields. |
303
- | `DeletableEdge` | Renders data connections; a selected edge can be deleted. |
304
 
305
  Device selection is always scoped to the current project. A new empty workflow does not display a
306
  device from another project. After data is uploaded, each card's selector shows only real sources
@@ -434,17 +436,17 @@ The build output is written to `web/static/workflow-studio/` and served by FastA
434
  Configuration is stored in `processor/.env` or environment variables and must not be
435
  committed:
436
 
437
- | Setting | Purpose |
438
- | --- | --- |
439
- | `STORAGE_DIR` | Root directory for data and system state |
440
- | `STORAGE_BACKEND` | `local` or `sftp` |
441
- | `SFTP_*` | Remote data directory and SSH connection settings |
442
- | `API_KEY` | API key for the capture client and protected endpoints |
443
- | `WORKER_API_KEY` | API key used by the Worker to claim and return jobs |
444
- | `JWT_SECRET` | Secret used to sign Web login sessions |
445
- | `UPLOAD_STAGING_DIR` | Local staging directory for upload archives |
446
- | `HOST` / `PORT` | Web/API listen address and port |
447
- | `PUBLIC_BASE_URL` | Public service URL |
448
 
449
  See [`.env.example`](.env.example) for the complete template. Production environments must use
450
  random secrets and restrict `.env` permissions. Never put passwords, tokens, API keys, or database
 
1
+  
2
+
3
  # processor
4
 
5
  [English](README.md) | [中文](README.zh-CN.md)
 
161
 
162
  ### Video sources
163
 
164
+ | Type | Example source name | Description |
165
+ | ------------ | ----------------------------------------- | ------------------------------ |
166
+ | Mono RGB | `head_rgb` | Regular RGB video |
167
+ | Metric depth | `head_depth` | Independent metric-depth video |
168
+ | Stereo RGB | `stereo_left_rgb`, `stereo_right_rgb` | Left and right RGB videos |
169
+ | Stereo depth | `stereo_depth` | Independent depth video |
170
 
171
  RGB and Depth are always stored as independent streams. A depth-only stream is never treated as
172
  RGB and is never duplicated alongside the RGB stream.
 
252
 
253
  #### Capture modules (Input)
254
 
255
+ | Frontend module | Inputs | Outputs | Purpose and rules |
256
+ | ----------------------- | ------ | -------------------------------------------------- | -------------------------------------------------------------------------------------------------- |
257
+ | `RGB Camera` | None | `RGB Video` | Mono RGB video. The device is selected from the current project's data; blank means auto-match. |
258
+ | `RGB-D Camera` | None | `RGB Video`, `Depth` | Emits RGB and real depth streams from the same device; Depth can connect only to a depth input. |
259
+ | `Stereo RGB Camera` | None | `Left RGB Video`, `Right RGB Video` | Stereo RGB input. Connecting one side creates the corresponding left/right relationship. |
260
+ | `Stereo RGB-D Camera` | None | `Left RGB Video`, `Right RGB Video`, `Depth` | Stereo RGB plus real depth; Depth is used by RGB-D 3D processing. |
261
+ | `Glove Sensor` | None | `Glove Sensor Data` | Pressure, joint, or glove sensor data; no video output. It can connect directly to quality review. |
262
 
263
  #### Video and keypoint processing modules (Process)
264
 
265
+ | Frontend module | Inputs | Outputs | Purpose and rules |
266
+ | ------------------------ | ------------------------ | -------------- | -------------------------------------------------------------------------------------------------------------------- |
267
+ | `Human Annotation` | `RGB Video` | `Annotation` | Frame-level manual annotation on RGB video. |
268
+ | `AI Annotation` | `RGB Video` | `Annotation` | Video/segment annotation through a local or API VLM; API is recommended for deployment. |
269
+ | `RGB_TO_2D_BareHand` | `RGB Video` | `Hand 2D` | Bare-hand 2D keypoints.`Spatial` is display-only and exports no metric 3D. |
270
+ | `RGB_TO_2D_BlackGlove` | `RGB Video` | `Hand 2D` | Black-glove 2D keypoints. Without Depth, it does not calculate metric 3D. |
271
+ | `RGB-D_3D_BareHand` | `RGB Video`, `Depth` | `Hand 3D` | Metric 3D bare-hand keypoints from RGB and real Depth; a missing branch is skipped. |
272
+ | `RGB-D_3D_BlackGlove` | `RGB Video`, `Depth` | `Hand 3D` | Metric 3D black-glove keypoints from RGB and real Depth; it does not search the disk or fall back to RGB estimation. |
273
 
274
  `MediaPipe Hand` remains available for historical workflows and backend compatibility but is hidden
275
  from the new-workflow palette. All nodes provide hover text describing their purpose and accepted
 
277
 
278
  #### Review modules (Review)
279
 
280
+ | Frontend module | Inputs | Outputs | Purpose and rules |
281
+ | --------------------- | ------------------------- | ----------------- | ------------------------------------------------------------------------------------------------------ |
282
+ | `Human Review` | `Review Target` | `Reviewed Data` | Human inspection of video, keypoints, or annotation results before downstream processing. |
283
  | `AI Quality Review` | `Quality Review Target` | `Reviewed Data` | Checks video decoding, frame continuity, black screens, freezes, annotation coverage, and sensor data. |
284
 
285
  #### Export modules (Export)
286
 
287
+ | Frontend module | Inputs | Outputs | Purpose and rules |
288
+ | ------------------ | ------------------- | ----------- | ------------------------------------------------------------------ |
289
  | `LeRobot Export` | `Exportable Data` | `Dataset` | Exports LeRobot v2.1 or v3.0; the version is selected on the node. |
290
+ | `HDF5 Export` | `Exportable Data` | `Dataset` | Exports HDF5 with the configured compression parameters. |
291
 
292
  #### Workflow Studio UI components
293
 
294
  <details>
295
  <summary>Click to expand: frontend component reference</summary>
296
 
297
+ | UI component | Function |
298
+ | --------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
299
+ | `NodePalette` | Displays Input, Process, Review, and Export categories; supports search, category collapse, and hover descriptions. |
300
+ | `WorkflowCanvas` | Supports drag-and-drop node creation, typed-port connections, node movement, selection, zoom, grid snapping, and minimap navigation. |
301
+ | `WorkflowNode` | Renders the card title, ports, project device selector/input,`Spatial` button, and API settings button. |
302
+ | `WorkflowDrawer` | Lists workflows, creates new workflows, loads workflows, and refreshes project-specific input sources. |
303
+ | `PipelineToolbar` | Provides New, Save, Save As, workflow JSON Export, and Run actions; the yellow dot indicates unsaved changes. |
304
+ | `NodeSettingsModal` | Configures AI Annotation API vendor, model, endpoint, and key fields. |
305
+ | `DeletableEdge` | Renders data connections; a selected edge can be deleted. |
306
 
307
  Device selection is always scoped to the current project. A new empty workflow does not display a
308
  device from another project. After data is uploaded, each card's selector shows only real sources
 
436
  Configuration is stored in `processor/.env` or environment variables and must not be
437
  committed:
438
 
439
+ | Setting | Purpose |
440
+ | ---------------------- | ------------------------------------------------------ |
441
+ | `STORAGE_DIR` | Root directory for data and system state |
442
+ | `STORAGE_BACKEND` | `local` or `sftp` |
443
+ | `SFTP_*` | Remote data directory and SSH connection settings |
444
+ | `API_KEY` | API key for the capture client and protected endpoints |
445
+ | `WORKER_API_KEY` | API key used by the Worker to claim and return jobs |
446
+ | `JWT_SECRET` | Secret used to sign Web login sessions |
447
+ | `UPLOAD_STAGING_DIR` | Local staging directory for upload archives |
448
+ | `HOST` / `PORT` | Web/API listen address and port |
449
+ | `PUBLIC_BASE_URL` | Public service URL |
450
 
451
  See [`.env.example`](.env.example) for the complete template. Production environments must use
452
  random secrets and restrict `.env` permissions. Never put passwords, tokens, API keys, or database
processor/app/api/worker.py CHANGED
@@ -22,7 +22,7 @@ from fastapi.responses import FileResponse, Response
22
 
23
  from app.config import settings
24
  from app.localstore import (
25
- list_runs, get_run, save_run, get_episode,
26
  set_episode_status, list_exceptions, delete_exception,
27
  update_run_if_owned, save_annotations,
28
  )
@@ -59,9 +59,11 @@ def _remove_later(path: str) -> None:
59
  async def claim_job(body: dict, _: str = Depends(verify_worker_api_key)):
60
  """Worker 轮询领取任务:queued 或租约过期的 running。"""
61
  now = _now()
62
- runs = list_runs()
 
 
63
  candidate = None
64
- for r in sorted(runs, key=lambda r: r.get("created_at") or ""):
65
  status = r.get("status")
66
  if status in ("queued", "pending"):
67
  candidate = r
@@ -78,7 +80,9 @@ async def claim_job(body: dict, _: str = Depends(verify_worker_api_key)):
78
  if candidate is None:
79
  return Response(status_code=204)
80
 
81
- run = candidate
 
 
82
  run["status"] = "running"
83
  run["worker_id"] = body.get("worker_id")
84
  run["lease_token"] = uuid.uuid4().hex
@@ -182,6 +186,30 @@ def clear_input_zip_cache(episode_id: str) -> None:
182
  pass
183
 
184
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
185
  def prepare_episode_input_cache(episode_id: str, batch_dir: Path | None = None) -> Path:
186
  """Build a local immutable input archive for a workflow batch.
187
 
@@ -201,16 +229,47 @@ def prepare_episode_input_cache(episode_id: str, batch_dir: Path | None = None)
201
 
202
  zip_path, stamp_path = _input_zip_cache_paths(str(episode_id))
203
  with _INPUT_ZIP_LOCK:
204
- if zip_path.is_file() and _input_zip_stamp_matches(stamp_path, batch_dir):
 
 
 
 
 
205
  return zip_path
206
 
207
- tmp = tempfile.NamedTemporaryFile(
208
- prefix=f"egodata-input-{episode_id}-", suffix=".zip",
209
- dir=str(_worker_tmp_root()), delete=False,
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
210
  )
211
- tmp.close()
212
- try:
213
- with zipfile.ZipFile(tmp.name, "w", allowZip64=True) as archive:
 
 
 
 
 
 
 
 
 
 
214
  canonical_episode = None
215
  if (batch_dir / "meta" / "episodes").is_dir():
216
  from app.project_dataset import episode_row, episode_files
@@ -252,21 +311,10 @@ def prepare_episode_input_cache(episode_id: str, batch_dir: Path | None = None)
252
  else zipfile.ZIP_DEFLATED
253
  )
254
  archive.write(path, str(arcname), compress_type=compression)
255
- except Exception:
256
- _remove_later(tmp.name)
257
- raise
258
-
259
- try:
260
- _input_zip_cache_dir().mkdir(parents=True, exist_ok=True)
261
- shutil.move(tmp.name, str(zip_path))
262
- stamp_path.write_text(
263
- json.dumps(_input_zip_stamp(batch_dir)), encoding="utf-8"
264
- )
265
- _evict_input_zip_cache()
266
- except Exception:
267
- _remove_later(tmp.name)
268
- raise
269
- return zip_path
270
 
271
 
272
  @router.get("/jobs/{run_id}/input")
@@ -410,6 +458,16 @@ async def complete_job(
410
  finally:
411
  shutil.rmtree(staging_dir, ignore_errors=True)
412
 
 
 
 
 
 
 
 
 
 
 
413
  def mark_completed(current: dict) -> None:
414
  current["status"] = "completed"
415
  current["finished_at"] = finished_at
 
22
 
23
  from app.config import settings
24
  from app.localstore import (
25
+ list_runs, list_run_summaries, get_run, save_run, get_episode,
26
  set_episode_status, list_exceptions, delete_exception,
27
  update_run_if_owned, save_annotations,
28
  )
 
59
  async def claim_job(body: dict, _: str = Depends(verify_worker_api_key)):
60
  """Worker 轮询领取任务:queued 或租约过期的 running。"""
61
  now = _now()
62
+ # Scan summaries, not full records: this endpoint is polled every few
63
+ # seconds and reading every historical run on each poll is what saturated
64
+ # the API process. Only the winner gets its full payload loaded.
65
  candidate = None
66
+ for r in sorted(list_run_summaries(), key=lambda r: r.get("created_at") or ""):
67
  status = r.get("status")
68
  if status in ("queued", "pending"):
69
  candidate = r
 
80
  if candidate is None:
81
  return Response(status_code=204)
82
 
83
+ # Reload the full record before mutating — save_run replaces the whole
84
+ # document, so writing the summary back would drop the run's payload.
85
+ run = get_run(str(candidate.get("id"))) or dict(candidate)
86
  run["status"] = "running"
87
  run["worker_id"] = body.get("worker_id")
88
  run["lease_token"] = uuid.uuid4().hex
 
186
  pass
187
 
188
 
189
+ def _zip_data_parquets_readable(archive_path: Path) -> bool:
190
+ """包内 ``data/*.parquet`` 是否都能读出 footer。
191
+
192
+ NAS/SSHFS 偶尔返回被截断的内容,而 ``archive.write`` 会把它原样打进包。
193
+ 这种包一旦缓存下来就会**永久**生效:worker 解压后读到的副本本身就是坏
194
+ 的,重试也救不回来 —— 表现为
195
+ ``ArrowInvalid: Parquet magic bytes not found in footer``。
196
+
197
+ 只读 schema(footer)不做全量解码,代价很小。
198
+ """
199
+ try:
200
+ import pyarrow.parquet as pq
201
+ with zipfile.ZipFile(archive_path) as archive:
202
+ for info in archive.infolist():
203
+ name = info.filename
204
+ if not (name.startswith("data/") and name.endswith(".parquet")):
205
+ continue
206
+ with archive.open(info) as handle:
207
+ pq.read_schema(handle)
208
+ return True
209
+ except Exception:
210
+ return False
211
+
212
+
213
  def prepare_episode_input_cache(episode_id: str, batch_dir: Path | None = None) -> Path:
214
  """Build a local immutable input archive for a workflow batch.
215
 
 
229
 
230
  zip_path, stamp_path = _input_zip_cache_paths(str(episode_id))
231
  with _INPUT_ZIP_LOCK:
232
+ # 缓存命中也要校验:戳只看 meta/info.json,而合并只重写 data parquet
233
+ # —— 戳因此可能在内容已变时仍判为新鲜,把早先固化的坏包一直发出去。
234
+ # 读 footer 很便宜,换掉「坏包只能用一次才知道」的被动局面。
235
+ if (zip_path.is_file()
236
+ and _input_zip_stamp_matches(stamp_path, batch_dir)
237
+ and _zip_data_parquets_readable(zip_path)):
238
  return zip_path
239
 
240
+ # 截断的网络读入不该被缓存:校验失败就重打,连续失败才报错。
241
+ # 打包本身很便宜(本地 zip),代价远低于把坏包固化进缓存。
242
+ packaged: str | None = None
243
+ for attempt in range(3):
244
+ packaged = _build_input_zip(episode_id, batch_dir)
245
+ if _zip_data_parquets_readable(Path(packaged)):
246
+ break
247
+ _remove_later(packaged)
248
+ packaged = None
249
+ print(f"[Worker] input archive unreadable, rebuilding "
250
+ f"({attempt + 1}/3): {episode_id}")
251
+ if packaged is None:
252
+ raise RuntimeError(
253
+ f"Input archive for {episode_id} stays unreadable after 3 attempts "
254
+ f"(source on the storage mount may be corrupt)")
255
+ _input_zip_cache_dir().mkdir(parents=True, exist_ok=True)
256
+ shutil.move(packaged, str(zip_path))
257
+ stamp_path.write_text(
258
+ json.dumps(_input_zip_stamp(batch_dir)), encoding="utf-8"
259
  )
260
+ _evict_input_zip_cache()
261
+ return zip_path
262
+
263
+
264
+ def _build_input_zip(episode_id: str, batch_dir: Path) -> str:
265
+ """把一集的 canonical 文件打成 zip,返回临时包路径。"""
266
+ tmp = tempfile.NamedTemporaryFile(
267
+ prefix=f"egodata-input-{episode_id}-", suffix=".zip",
268
+ dir=str(_worker_tmp_root()), delete=False,
269
+ )
270
+ tmp.close()
271
+ try:
272
+ with zipfile.ZipFile(tmp.name, "w", allowZip64=True) as archive:
273
  canonical_episode = None
274
  if (batch_dir / "meta" / "episodes").is_dir():
275
  from app.project_dataset import episode_row, episode_files
 
311
  else zipfile.ZIP_DEFLATED
312
  )
313
  archive.write(path, str(arcname), compress_type=compression)
314
+ except Exception:
315
+ _remove_later(tmp.name)
316
+ raise
317
+ return tmp.name
 
 
 
 
 
 
 
 
 
 
 
318
 
319
 
320
  @router.get("/jobs/{run_id}/input")
 
458
  finally:
459
  shutil.rmtree(staging_dir, ignore_errors=True)
460
 
461
+ # 合并刚重写了该集的 data parquet,而 Worker 输入包的「新鲜度戳」只看
462
+ # meta/info.json —— 合并不碰那个文件,戳因此不变,坏包会被一直复用。
463
+ # 实测后果:包内 parquet 被截断(ArrowInvalid: Parquet magic bytes not
464
+ # found in footer),节点每次都以「列不可读」跳过,且重试无用(解压出来的
465
+ # 副本本身就是坏的)。这里主动清掉,下一次运行会重新打包。
466
+ try:
467
+ clear_input_zip_cache(str(run.get("episode_id") or ""))
468
+ except Exception:
469
+ pass # 清缓存失败不影响本次结果,下次打包仍会走戳校验
470
+
471
  def mark_completed(current: dict) -> None:
472
  current["status"] = "completed"
473
  current["finished_at"] = finished_at
processor/app/batch_fingerprint.py ADDED
@@ -0,0 +1,194 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """批次内容指纹 — 识别"换个包名重传同一份数据"。
2
+
3
+ 采集端偶尔会把已经上传过的录制再推一遍,但**包名变了**(例如重装、
4
+ 重开 App 后本地去重标记丢失,或包名规则从序号变成时间戳)。此时
5
+ ``session.py`` 的 ``incoming_name in existing_ids`` 判不出重复,同一份
6
+ 数据就会被当成新批次追加,审核页出现两条一模一样的数据,导出也会
7
+ 重复计一份(实测:UMIGripper_AI 的 4 组批次视频字节级相同、
8
+ parquet 除 ``episode_index`` 外逐值相同)。
9
+
10
+ 本模块给出一个与**包名无关**的内容身份:
11
+
12
+ - 每个视频文件的 md5(按 source_key 排序)
13
+ - canonical data parquet 的内容哈希(排除 ``episode_index`` ——
14
+ 同一次录制被追加成不同 episode 时,只有这一列会变)
15
+
16
+ 合成为一个短字符串 key,存到 ``state/batch_fingerprints.json``。
17
+ 上传时先算新批次的 key,再按项目查表:命中且目标 episode 仍然存在,
18
+ 就判定为重复上传。
19
+
20
+ **失败一律放行**:算不出指纹、索引读不到、目标 episode 已被删除 ——
21
+ 任一情况都按"新批次"正常入库(即本模块不存在时的行为),绝不因为
22
+ 指纹机制本身的问题丢数据。
23
+ """
24
+
25
+ from __future__ import annotations
26
+
27
+ import hashlib
28
+ import json
29
+ import threading
30
+ from pathlib import Path
31
+ from typing import Any
32
+
33
+ from app.localstore import STATE_ROOT
34
+
35
+ INDEX_PATH = STATE_ROOT / "batch_fingerprints.json"
36
+ _LOCK = threading.Lock()
37
+
38
+ # 视频按块读;读取量固定,不随文件大小膨胀。
39
+ _CHUNK = 1 << 20
40
+
41
+
42
+ def _file_md5(path: Path) -> str | None:
43
+ digest = hashlib.md5()
44
+ try:
45
+ with open(path, "rb") as handle:
46
+ for block in iter(lambda: handle.read(_CHUNK), b""):
47
+ digest.update(block)
48
+ except OSError:
49
+ return None
50
+ return digest.hexdigest()
51
+
52
+
53
+ def _parquet_content_hash(path: Path) -> str | None:
54
+ """canonical data parquet 的内容哈希,排除 ``episode_index``。
55
+
56
+ ``episode_index`` 是同一次录制被追加成不同 episode 时唯一会变的列,
57
+ 必须排除,否则重复上传算不出相同的 key。用 Arrow IPC 序列化取字节:
58
+ 对嵌套 list 列(手套 250x250x3 力矩阵、手部关键点)比对 pandas 的
59
+ hash 更稳,且与行序无关的假设更少 —— 这里行序本身就是内容的一部分。
60
+ """
61
+ try:
62
+ import pyarrow as pa
63
+ import pyarrow.parquet as pq
64
+ except ImportError:
65
+ return None
66
+ try:
67
+ table = pq.read_table(path)
68
+ if "episode_index" in table.column_names:
69
+ table = table.drop_columns(["episode_index"])
70
+ sink = pa.BufferOutputStream()
71
+ with pa.ipc.new_stream(sink, table.schema) as writer:
72
+ writer.write_table(table)
73
+ return hashlib.md5(sink.getvalue().to_pybytes()).hexdigest()
74
+ except Exception:
75
+ return None
76
+
77
+
78
+ def compute_fingerprint(root: Path) -> str | None:
79
+ """Staging 目录(已 normalize)的内容指纹;算不出返回 None。"""
80
+ root = Path(root)
81
+ parts: list[str] = []
82
+
83
+ videos_root = root / "videos"
84
+ if videos_root.is_dir():
85
+ from app.lerobot_v21 import iter_video_streams
86
+
87
+ entries: list[tuple[str, str]] = []
88
+ for source, path in iter_video_streams(videos_root):
89
+ digest = _file_md5(Path(path))
90
+ if digest is None:
91
+ return None
92
+ entries.append((str(source), digest))
93
+ if not entries:
94
+ return None
95
+ parts.append("|".join(f"v:{s}={d}" for s, d in sorted(entries)))
96
+
97
+ data_files = sorted(
98
+ path for path in (root / "data").rglob("*.parquet")
99
+ if path.is_file() and "/meta/" not in path.as_posix().casefold()
100
+ )
101
+ if not data_files:
102
+ return None
103
+ digest = _parquet_content_hash(data_files[0])
104
+ if digest is None:
105
+ return None
106
+ parts.append(f"d:{digest}")
107
+
108
+ return ";".join(parts)
109
+
110
+
111
+ def _read_index() -> dict[str, dict[str, str]]:
112
+ try:
113
+ value = json.loads(INDEX_PATH.read_text(encoding="utf-8"))
114
+ except (OSError, ValueError):
115
+ return {}
116
+ return value if isinstance(value, dict) else {}
117
+
118
+
119
+ def _write_index(index: dict) -> None:
120
+ try:
121
+ INDEX_PATH.parent.mkdir(parents=True, exist_ok=True)
122
+ temporary = INDEX_PATH.with_suffix(".json.tmp")
123
+ temporary.write_text(
124
+ json.dumps(index, ensure_ascii=False, indent=2), encoding="utf-8")
125
+ temporary.replace(INDEX_PATH)
126
+ except OSError:
127
+ pass
128
+
129
+
130
+ def lookup_duplicate(project_folder: str, fingerprint: str | None,
131
+ live_ids: set[str]) -> str | None:
132
+ """返回与本次上传内容相同的、且**仍然存在**的 episode_id。
133
+
134
+ ``live_ids`` 来自当前项目数据集的实际扫描结果:索引里指向已删除
135
+ episode 的陈旧条目在这里被自然过滤掉,因此"删除后重新上传同一条
136
+ 录制"仍会正常入库,不会被误判成重复。
137
+ """
138
+ if not fingerprint:
139
+ return None
140
+ with _LOCK:
141
+ index = _read_index()
142
+ entries = index.get(str(project_folder))
143
+ if not isinstance(entries, dict):
144
+ return None
145
+ episode_id = entries.get(str(fingerprint))
146
+ if not episode_id:
147
+ return None
148
+ return str(episode_id) if str(episode_id) in live_ids else None
149
+
150
+
151
+ def record_fingerprint(project_folder: str, fingerprint: str | None,
152
+ episode_id: str) -> None:
153
+ """记录已提交批次的内容指纹(失败静默:只是失去一次去重机会)。"""
154
+ if not fingerprint:
155
+ return
156
+ with _LOCK:
157
+ index = _read_index()
158
+ if not isinstance(index, dict):
159
+ index = {}
160
+ entries = index.get(str(project_folder))
161
+ if not isinstance(entries, dict):
162
+ entries = {}
163
+ entries[str(fingerprint)] = str(episode_id)
164
+ # 同一 episode 换内容(同名重传)时清掉它名下的旧指纹,避免
165
+ # 索引里留下永远指不到实际内容的条目。
166
+ for key in [k for k, v in entries.items()
167
+ if str(v) == str(episode_id) and k != str(fingerprint)]:
168
+ entries.pop(key, None)
169
+ index[str(project_folder)] = entries
170
+ _write_index(index)
171
+
172
+
173
+ def forget_fingerprint(episode_id: str) -> None:
174
+ """批次被删除后清除其指纹条目。"""
175
+ target = str(episode_id)
176
+ with _LOCK:
177
+ index = _read_index()
178
+ changed = False
179
+ for entries in index.values():
180
+ if not isinstance(entries, dict):
181
+ continue
182
+ for key in [k for k, v in entries.items() if str(v) == target]:
183
+ entries.pop(key, None)
184
+ changed = True
185
+ if changed:
186
+ _write_index(index)
187
+
188
+
189
+ def describe(project_folder: str) -> dict[str, Any]:
190
+ """诊断用:该项目已记录的内容指纹。"""
191
+ with _LOCK:
192
+ index = _read_index()
193
+ entries = index.get(str(project_folder))
194
+ return entries if isinstance(entries, dict) else {}
processor/app/device_progress.py ADDED
@@ -0,0 +1,218 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """采集端任务进度上报账本 —— 本地 JSON 存储(无数据库)。
2
+
3
+ 为什么要这个账本:``GET /device/tasks`` 原先把「项目目录下的 session 数」当
4
+ 采集进度,而 session 是**上传**的产物。采集端本身在「录制完成」时就已经把
5
+ 增量上报上来(见采集端 ``core/task_record.py`` 的分账模型),于是同一条数据
6
+ 被算两次——录制一次、上传一次。2026-09-10 现场:UMIGripper_AI 本机录了 12 条、
7
+ 服务器上 16 个 session,界面显示 28/20,两边的数都不可信。
8
+
9
+ 现在的口径:
10
+
11
+ 项目进度 = 各设备上报的录制完成数之和
12
+
13
+ 只有**从未收到过上报**的项目才回退到 session 计数,这样升级前就存在的历史
14
+ 项目不会突然归零;某个项目一旦有设备上报,它就固定按上报口径走(上传不再
15
+ 改变它,采集端录一条 +1)。
16
+
17
+ 幂等:采集端用 ``session_id = "{device}:{task_id}:{水位}"`` 标识一次增量,
18
+ 同一 (task_id, device_name, session_id) 只计一次 —— 网络抖动重试、程序被杀
19
+ 后重放同一个水位,都不会重复计数。
20
+
21
+ 文件: ``data/state/device_task_progress.json``
22
+
23
+ {"tasks": {"<task_id>": {
24
+ "devices": {"<device_name>": {
25
+ "count": 12,
26
+ "sessions": ["EGO_001:<task_id>:0", ...], # 已计入的幂等键
27
+ "updated_at": "2026-09-10T02:11:39+00:00"}},
28
+ "updated_at": "..."}},
29
+ "updated_at": "..."}
30
+ """
31
+
32
+ from __future__ import annotations
33
+
34
+ import json
35
+ import threading
36
+ from datetime import datetime, timezone
37
+ from pathlib import Path
38
+ from typing import Iterable
39
+
40
+ # 读-改-写必须串行:uvicorn 单进程多线程,两个设备同时上报时不能丢增量
41
+ _LOCK = threading.RLock()
42
+
43
+ PROGRESS_FILE_NAME = "device_task_progress.json"
44
+
45
+ # 单次上报上限:采集端 flush 的是「本机未同步增量」,正常是 1~几十;
46
+ # 超过这个数说明上报方数据脏了,直接拒绝而不是把进度顶飞。
47
+ MAX_INCREMENT = 100000
48
+
49
+ _MAX_FIELD = 200 # task_id / device_name / session_id 的长度上限
50
+
51
+
52
+ def _progress_file(path: Path | None = None) -> Path:
53
+ if path is not None:
54
+ return Path(path)
55
+ from app.localstore import STATE_ROOT
56
+ return STATE_ROOT / PROGRESS_FILE_NAME
57
+
58
+
59
+ def _now() -> str:
60
+ return datetime.now(timezone.utc).isoformat()
61
+
62
+
63
+ def _read(path: Path | None = None) -> dict:
64
+ try:
65
+ data = json.loads(_progress_file(path).read_text(encoding="utf-8"))
66
+ except Exception:
67
+ return {"tasks": {}}
68
+ if not isinstance(data, dict):
69
+ return {"tasks": {}}
70
+ tasks = data.get("tasks")
71
+ if not isinstance(tasks, dict):
72
+ data["tasks"] = {}
73
+ return data
74
+
75
+
76
+ def _write(data: dict, path: Path | None = None) -> None:
77
+ target = _progress_file(path)
78
+ target.parent.mkdir(parents=True, exist_ok=True)
79
+ tmp = target.with_suffix(target.suffix + ".tmp")
80
+ tmp.write_text(json.dumps(data, ensure_ascii=False, indent=1), encoding="utf-8")
81
+ tmp.replace(target)
82
+
83
+
84
+ def _clean(value, limit: int = _MAX_FIELD) -> str:
85
+ return str(value or "").strip()[:limit]
86
+
87
+
88
+ def reported_totals(path: Path | None = None) -> dict[str, int]:
89
+ """各任务的上报聚合数。**只含收到过上报的任务** —— 没上报过的项目不在
90
+ 这里,调用方据此回退到 session 计数。"""
91
+ out: dict[str, int] = {}
92
+ with _LOCK:
93
+ data = _read(path)
94
+ for task_id, entry in (data.get("tasks") or {}).items():
95
+ if not isinstance(entry, dict):
96
+ continue
97
+ total = 0
98
+ for dev in (entry.get("devices") or {}).values():
99
+ if isinstance(dev, dict):
100
+ total += int(dev.get("count") or 0)
101
+ out[str(task_id)] = total
102
+ return out
103
+
104
+
105
+ def reported_total(task_id: str, path: Path | None = None) -> int | None:
106
+ """单个任务的上报聚合数;从未上报过返回 None。"""
107
+ return reported_totals(path).get(str(task_id))
108
+
109
+
110
+ def needs_session_fallback(projects: list[dict],
111
+ reported: dict[str, int]) -> bool:
112
+ """是否还有项目需要 session 计数兜底。
113
+
114
+ 全都有上报时调用方可跳过目录扫描(远程存储上的一次全量遍历)。
115
+ """
116
+ return any(str(p.get("id") or "") not in reported for p in projects)
117
+
118
+
119
+ def resolve_count(project: dict, reported: dict[str, int],
120
+ sessions: int) -> tuple[int, str]:
121
+ """单个项目的 (current_count, 口径)。
122
+
123
+ 有上报 → 上报聚合数(录制口径,上传不再影响它);
124
+ 没有 → session 数(升级前的历史项目不归零)。
125
+ """
126
+ pid = str(project.get("id") or "")
127
+ if pid in reported:
128
+ return int(reported[pid]), "reported"
129
+ return int(sessions or 0), "sessions"
130
+
131
+
132
+ def apply_increment(task_id: str, device_name: str, session_id: str,
133
+ increment: int, path: Path | None = None) -> dict:
134
+ """把一次上报落账,返回该任务的最新聚合数。
135
+
136
+ 幂等:同一 (task_id, device_name, session_id) 第二次进来只读取不累加
137
+ (``applied`` 为 False),采集端重试因此安全。
138
+ """
139
+ task_id = _clean(task_id)
140
+ device_name = _clean(device_name)
141
+ session_id = _clean(session_id)
142
+ with _LOCK:
143
+ data = _read(path)
144
+ tasks = data.setdefault("tasks", {})
145
+ entry = tasks.setdefault(task_id, {"devices": {}})
146
+ if not isinstance(entry.get("devices"), dict):
147
+ entry["devices"] = {}
148
+ dev = entry["devices"].setdefault(
149
+ device_name, {"count": 0, "sessions": []})
150
+ sessions = dev.get("sessions")
151
+ if not isinstance(sessions, list):
152
+ sessions = dev["sessions"] = []
153
+ applied = False
154
+ if session_id not in sessions:
155
+ dev["count"] = int(dev.get("count") or 0) + int(increment)
156
+ sessions.append(session_id)
157
+ dev["updated_at"] = _now()
158
+ entry["updated_at"] = dev["updated_at"]
159
+ data["updated_at"] = dev["updated_at"]
160
+ applied = True
161
+ total = sum(int(d.get("count") or 0)
162
+ for d in entry["devices"].values() if isinstance(d, dict))
163
+ if applied:
164
+ _write(data, path)
165
+ return {"total": total, "applied": applied,
166
+ "device_count": int(dev.get("count") or 0),
167
+ "device": device_name, "task_id": task_id}
168
+
169
+
170
+ def apply_report(body: dict, known_task_ids: Iterable[str] | None = None,
171
+ path: Path | None = None) -> dict:
172
+ """校验并落账一次上报,返回响应体(路由直接回给采集端)。
173
+
174
+ Raises:
175
+ ValueError: 参数缺失/非法/任务不存在 —— 调用方转 HTTP 400。
176
+
177
+ 刻意**不返回 404**:采集端把 404 当成「后端还没实现该端点」并永久降级
178
+ (``_progress_supported = False``),此后整机所有任务都不再上报。参数错
179
+ 与未知任务都属于「这次请求有问题」,用 400 表达即可。
180
+ """
181
+ if not isinstance(body, dict):
182
+ raise ValueError("body must be a JSON object")
183
+ task_id = _clean(body.get("task_id"))
184
+ device_name = _clean(body.get("device_name"))
185
+ session_id = _clean(body.get("session_id"))
186
+ if not task_id:
187
+ raise ValueError("task_id required")
188
+ if not device_name:
189
+ raise ValueError("device_name required")
190
+ if not session_id:
191
+ raise ValueError("session_id required (idempotency key)")
192
+
193
+ raw = body.get("increment", 0)
194
+ if isinstance(raw, bool):
195
+ raise ValueError("increment must be an integer")
196
+ try:
197
+ increment = int(raw)
198
+ except (TypeError, ValueError):
199
+ raise ValueError("increment must be an integer")
200
+ if increment < 0:
201
+ raise ValueError("increment must be >= 0")
202
+ if increment > MAX_INCREMENT:
203
+ raise ValueError(f"increment too large (max {MAX_INCREMENT})")
204
+
205
+ if known_task_ids is not None and task_id not in set(known_task_ids):
206
+ raise ValueError(f"unknown task_id: {task_id}")
207
+
208
+ result = apply_increment(task_id, device_name, session_id, increment, path)
209
+ return {
210
+ "ok": True,
211
+ "task_id": task_id,
212
+ "device_name": device_name,
213
+ "applied": result["applied"],
214
+ "increment": increment,
215
+ # 采集端读这个字段当后端权威数 —— 必须是 int(布尔会被 isinstance 放行)
216
+ "completed_count": int(result["total"]),
217
+ "updated_at": _now(),
218
+ }
processor/app/lerobot_export.py CHANGED
@@ -129,9 +129,11 @@ def _write_stats(output_dir: Path,
129
  if not np.isfinite(num).any():
130
  continue
131
  q = _quantiles(num)
 
 
 
 
132
  stats[col] = {
133
- "dtype": "float32" if np.issubdtype(s.dtype, np.floating) else "int64",
134
- "shape": [1],
135
  "mean": float(np.nanmean(num)),
136
  "std": float(np.nanstd(num)),
137
  "min": float(np.nanmin(num)),
@@ -268,7 +270,22 @@ def _write_stats(output_dir: Path,
268
  }
269
  continue
270
 
271
- samples: list = []
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
272
  for vpath in videos:
273
  cap = cv2.VideoCapture(str(vpath))
274
  if not cap.isOpened():
@@ -283,25 +300,44 @@ def _write_stats(output_dir: Path,
283
  if not ok:
284
  break
285
  if read_idx % step == 0 and frame.ndim == 3:
286
- samples.append(frame.astype(np.float32)[..., :channels] / 255.0)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
287
  read_idx += 1
288
  cap.release()
289
- if not samples:
290
  continue
291
- stack = np.stack(samples) # (T, H, W, C)
292
- per_channel = stack.reshape(-1, stack.shape[-1])
293
  def _nested(vals):
294
  return [[[float(v)] for v in vals]]
295
- q = np.percentile(per_channel, [1, 10, 50, 90, 99], axis=0)
 
 
 
 
 
 
296
  stats[vkey] = {
297
- "min": _nested(per_channel.min(axis=0)),
298
- "max": _nested(per_channel.max(axis=0)),
299
- "mean": _nested(per_channel.mean(axis=0)),
300
- "std": _nested(per_channel.std(axis=0)),
301
  "q01": _nested(q[0]), "q10": _nested(q[1]),
302
  "q50": _nested(q[2]), "q90": _nested(q[3]),
303
  "q99": _nested(q[4]),
304
- "count": [int(per_channel.shape[0] * per_channel.shape[1])],
305
  }
306
  except Exception:
307
  pass
@@ -686,8 +722,14 @@ _TACTILE_COLUMN_MAP = {
686
 
687
 
688
  def _read_sensor_rows(session_dir: Path,
689
- episode_id: str | None = None) -> dict[int, dict]:
690
- """Merge independently stored glove/action parquet rows by frame_index."""
 
 
 
 
 
 
691
  rows: dict[int, dict] = {}
692
  try:
693
  import pyarrow.parquet as pq
@@ -705,7 +747,13 @@ def _read_sensor_rows(session_dir: Path,
705
  and path.name != f"episode_{episode_index:06d}.parquet"):
706
  continue
707
  try:
708
- table = pq.read_table(path)
 
 
 
 
 
 
709
  names = table.schema.names
710
  if "frame_index" not in names:
711
  continue
@@ -1059,16 +1107,221 @@ def _passthrough_dtype(key: str, sensor_rows: dict[int, dict]) -> str:
1059
  return "float32"
1060
 
1061
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1062
  def _pad_variable_columns(rows: list[dict],
1063
- nan_fill: float = float("nan")) -> int:
1064
- """IMU 变长列固定长度化(官方加载器不支持变长:list<int64> 对声明
1065
- int64[1] 实测 cast 失败)。返回补齐后的每帧样本数 cap。
1066
 
1067
  - observation.imu:每样本 6 维 → 展平为 cap*6(nan_fill 补尾)
1068
  - observation.imu_ts_ns:补 0 到 cap
1069
- 无 IMU 列时返回 3(不影响任何声明)。
 
 
 
 
1070
  """
1071
  import numpy as np
 
1072
  imu_cap = 3
1073
  for row in rows:
1074
  for key in ("observation.imu", "observation.imu_ts_ns"):
@@ -1076,7 +1329,7 @@ def _pad_variable_columns(rows: list[dict],
1076
  if isinstance(value, (list, tuple)):
1077
  imu_cap = max(imu_cap, len(value))
1078
  if not any("observation.imu" in row for row in rows):
1079
- return imu_cap
1080
  for row in rows:
1081
  imu = row.get("observation.imu")
1082
  if isinstance(imu, (list, tuple)):
@@ -1100,7 +1353,7 @@ def _pad_variable_columns(rows: list[dict],
1100
  if isinstance(ts, (list, tuple)):
1101
  row["observation.imu_ts_ns"] = (
1102
  [int(v) for v in ts] + [0] * (imu_cap - len(ts)))[:imu_cap]
1103
- return imu_cap
1104
 
1105
 
1106
  def _propagated_hand_sets(source_row: dict) -> tuple[set[str], set[str], dict[str, set[str]]]:
@@ -1662,9 +1915,15 @@ def build_lerobot_dataset(dataset_name: str, episode_ids: list[str], output_dir:
1662
  hand_3d_right_paths: list[str] | None = None,
1663
  version: str = "v3.0",
1664
  hand_3d_unit: str | None = None,
 
1665
  progress_callback=None) -> Path:
1666
  """Build a LeRobot dataset (``version`` = "v2.1" | "v3.0").
1667
 
 
 
 
 
 
1668
  Depth sources are exported as their preview video (``is_depth_map``)
1669
  plus the per-frame ``observation.depth.<source>.valid`` marker; the
1670
  original 16-bit PNG sequence stays in the session directory and is not
@@ -1749,7 +2008,9 @@ def build_lerobot_dataset(dataset_name: str, episode_ids: list[str], output_dir:
1749
  # 本集实际导出的每路视频(帧数, fps) → episodes parquet 的
1750
  # videos/<key>/from_timestamp 映射(官方 v3 加载器按时间戳定位帧)
1751
  episode_video_meta: dict[str, tuple[int, float]] = {}
1752
- sensor_rows = _read_sensor_rows(session_dir, episode_id)
 
 
1753
  annotation_map, annotation_defs, _skipped_candidates = _annotation_maps(episode_id)
1754
 
1755
  description = str(episode.get("project") or "").strip()
@@ -1791,7 +2052,7 @@ def build_lerobot_dataset(dataset_name: str, episode_ids: list[str], output_dir:
1791
  / f"observation.images.{source_key}")
1792
  output_video = destination / f"file-{out_episode_index:03d}.mp4"
1793
  destination.mkdir(parents=True, exist_ok=True)
1794
- shutil.copy2(source, output_video)
1795
  total_videos += 1
1796
  probed_count, probed_fps, width, height = _probe_video(source)
1797
  episode_video_meta[f"observation.images.{source_key}"] = (
@@ -2025,10 +2286,33 @@ def build_lerobot_dataset(dataset_name: str, episode_ids: list[str], output_dir:
2025
  except Exception:
2026
  pass # 进度回调失败不影响导出本身
2027
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2028
  # 变长异步传感器列固定长度化:官方加载器不支持变长列(实测
2029
  # list<int64> 对声明 int64[1] 直接 cast 失败)。取全数据集最大
2030
  # 样本数补齐,声明 shape 同步为固定长度,变长语义保留在 note。
2031
- imu_cap = _pad_variable_columns(
2032
  rows, nan_fill=0.0 if is_v2 else float("nan"))
2033
 
2034
  # 收敛:删恒定哨兵垃圾列(全程 NaN/-1/0/空串 → 数据从未写入)。
@@ -2301,6 +2585,11 @@ def build_lerobot_dataset(dataset_name: str, episode_ids: list[str], output_dir:
2301
  import numpy as np
2302
  arr = np.asarray(value)
2303
  shape = list(arr.shape) if arr.ndim else [1]
 
 
 
 
 
2304
  if key in ("observation.tactile.left", "observation.tactile.right"):
2305
  features[key] = {
2306
  "dtype": "float32", "shape": [256],
@@ -2341,6 +2630,33 @@ def build_lerobot_dataset(dataset_name: str, episode_ids: list[str], output_dir:
2341
  "shape": shape}
2342
  if junk_cols:
2343
  features = {k: v for k, v in features.items() if k not in junk_cols}
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2344
  # 深度有效帧标记列(行循环写入,不声明会导致官方加载器读不完整)
2345
  for depth_key in sorted(depth_source_keys):
2346
  key = f"observation.depth.{depth_key}.valid"
 
129
  if not np.isfinite(num).any():
130
  continue
131
  q = _quantiles(num)
132
+ # 只写数值统计量。不要再加 dtype/shape 之类的元数据字段:官方
133
+ # 加载器会把 stats 里的每一项都转成张量做归一化,字符串字段直接
134
+ # 崩在 torch.from_numpy(np.ndarray of type numpy.str_)。官方数据集
135
+ # 的 stats 也只有 min/max/mean/std/count。
136
  stats[col] = {
 
 
137
  "mean": float(np.nanmean(num)),
138
  "std": float(np.nanstd(num)),
139
  "min": float(np.nanmin(num)),
 
270
  }
271
  continue
272
 
273
+ # 增量累积:全部集的采样帧若一次性留在内存里会爆 —— 960×1280
274
+ # 的一帧 float32 是 14.75MB,34 集 × 约 200 采样帧 ≈ 100GB。
275
+ # 这里只保留跑动统计量(min/max/和/平方和/计数)与一个固定大小的
276
+ # 分位数蓄水池,内存与集数无关。
277
+ ch_sum = np.zeros(channels, dtype=np.float64)
278
+ ch_sq = np.zeros(channels, dtype=np.float64)
279
+ ch_min = np.full(channels, np.inf, dtype=np.float64)
280
+ ch_max = np.full(channels, -np.inf, dtype=np.float64)
281
+ pixel_count = 0
282
+ # 蓄水池上限与每帧贡献:分位数只需近似,但样本必须**跨帧、跨集**
283
+ # 均匀铺开 —— 只取前几帧会让分位数偏向最早的那一集。每帧按固定
284
+ # 步长抽约 1200 个像素,800 万像素上限可覆盖数千帧。
285
+ reservoir_cap = 8_000_000
286
+ per_frame_budget = 1200
287
+ taken = 0
288
+ reservoir: list = []
289
  for vpath in videos:
290
  cap = cv2.VideoCapture(str(vpath))
291
  if not cap.isOpened():
 
300
  if not ok:
301
  break
302
  if read_idx % step == 0 and frame.ndim == 3:
303
+ # 先按步长抽样再转 float:一帧 960×1280 有 120 万像素,
304
+ # 对整帧做 astype(float32)/255 是这一步的主要开销,
305
+ # 而统计只需要千级样本。抽样后再运算快约三个数量级。
306
+ flat_u8 = frame[..., :channels].reshape(-1, channels)
307
+ stride_u8 = max(1, flat_u8.shape[0] // per_frame_budget)
308
+ flat = (flat_u8[::stride_u8].astype(np.float32) / 255.0)
309
+ ch_sum += flat.sum(axis=0, dtype=np.float64)
310
+ ch_sq += np.square(flat, dtype=np.float64).sum(axis=0)
311
+ ch_min = np.minimum(ch_min, flat.min(axis=0))
312
+ ch_max = np.maximum(ch_max, flat.max(axis=0))
313
+ pixel_count += flat.shape[0]
314
+ if taken < reservoir_cap:
315
+ # flat 已是抽样结果,直接入池
316
+ take = min(reservoir_cap - taken, flat.shape[0])
317
+ reservoir.append(flat[:take].copy())
318
+ taken += take
319
  read_idx += 1
320
  cap.release()
321
+ if not pixel_count:
322
  continue
 
 
323
  def _nested(vals):
324
  return [[[float(v)] for v in vals]]
325
+ mean = ch_sum / pixel_count
326
+ var = np.maximum(ch_sq / pixel_count - mean * mean, 0.0)
327
+ pooled = np.concatenate(reservoir, axis=0) if reservoir else None
328
+ if pooled is not None and pooled.shape[0]:
329
+ q = np.percentile(pooled, [1, 10, 50, 90, 99], axis=0)
330
+ else:
331
+ q = np.stack([mean] * 5)
332
  stats[vkey] = {
333
+ "min": _nested(ch_min),
334
+ "max": _nested(ch_max),
335
+ "mean": _nested(mean),
336
+ "std": _nested(np.sqrt(var)),
337
  "q01": _nested(q[0]), "q10": _nested(q[1]),
338
  "q50": _nested(q[2]), "q90": _nested(q[3]),
339
  "q99": _nested(q[4]),
340
+ "count": [int(pixel_count)],
341
  }
342
  except Exception:
343
  pass
 
722
 
723
 
724
  def _read_sensor_rows(session_dir: Path,
725
+ episode_id: str | None = None,
726
+ skip_columns: set[str] | None = None) -> dict[int, dict]:
727
+ """Merge independently stored glove/action parquet rows by frame_index.
728
+
729
+ ``skip_columns``: 调用方已排除的列。这些列在读取阶段就被跳过 ——
730
+ 触觉力矩阵每帧 0.8MB,整集 14MB,在 SSHFS 上读完要十几秒;既然最终
731
+ 不会写进数据集,就不该读。仅按列名裁剪,不改变任何保留列的值。
732
+ """
733
  rows: dict[int, dict] = {}
734
  try:
735
  import pyarrow.parquet as pq
 
747
  and path.name != f"episode_{episode_index:06d}.parquet"):
748
  continue
749
  try:
750
+ if skip_columns:
751
+ # 先读 schema(只读 footer,便宜),再只取需要的列。
752
+ available = [str(n) for n in pq.read_schema(path).names]
753
+ wanted = [n for n in available if n not in skip_columns]
754
+ table = pq.read_table(path, columns=wanted)
755
+ else:
756
+ table = pq.read_table(path)
757
  names = table.schema.names
758
  if "frame_index" not in names:
759
  continue
 
1107
  return "float32"
1108
 
1109
 
1110
+ def _write_video_dense_keyframes(source: Path, destination: Path) -> None:
1111
+ """把视频写入数据集,必要时重编码以加密关键帧。
1112
+
1113
+ 采集端用 ffmpeg 默认设置编码(GOP=250),10 秒的集只有 2 个关键帧
1114
+ (0s 和 8.3s)。lerobot 按时间戳取帧时先 seek 到关键帧再向前解码,当
1115
+ 查询点紧邻下一个关键帧时 seek 会落到**目标之后**,该帧永远取不到,
1116
+ 训练直接抛 FrameTimestampError(实测 7 个采样点里坏 2 个)。
1117
+
1118
+ GOP=30(每秒一个关键帧)可让 seek 稳定落在目标之前,实测 7/7 全通过;
1119
+ 代价是文件约大 2 倍(2.7MB -> 5.7MB/集)。
1120
+
1121
+ 编码失败(缺 ffmpeg、编码器不可用)时退回原样复制:宁可保留稀疏关键帧
1122
+ 也不要让整个导出失败 —— 取不到的帧会让那条样本报错,而不是产出错数据。
1123
+ """
1124
+ import subprocess
1125
+ try:
1126
+ result = subprocess.run(
1127
+ ["ffmpeg", "-y", "-v", "error", "-i", str(source),
1128
+ "-c:v", "libx264", "-g", "30", "-pix_fmt", "yuv420p",
1129
+ "-crf", "20", "-preset", "veryfast", str(destination)],
1130
+ capture_output=True, timeout=300)
1131
+ if result.returncode == 0 and destination.is_file() and destination.stat().st_size:
1132
+ return
1133
+ except (OSError, subprocess.SubprocessError):
1134
+ pass
1135
+ shutil.copy2(source, destination)
1136
+
1137
+
1138
+ def _ensure_observation_state(rows: list[dict]) -> str | None:
1139
+ """确保每帧有 ``observation.state``;缺失时由 UMI 列派生。
1140
+
1141
+ ACT 的 VAE 编码器无条件读取 ``batch[observation.state]``
1142
+ (modeling_act.py 里那句 ``device=batch[OBS_STATE].device`` 没做
1143
+ robot_state_feature 判空),缺这列直接 KeyError。即便抛开这个上游
1144
+ 问题,单帧观测的 ACT 也需要 proprioception 才知道夹爪在哪。
1145
+
1146
+ 派生口径 ``[Δx, Δy, Δz, gripper]``:位置是本帧相对**本集第一帧**的
1147
+ 偏移。SLAM 的世界原点是每次采集初始化时任意确定的,直接喂绝对坐标
1148
+ 等于把无关的世界位置当特征。夹爪取开合度 0–1。
1149
+
1150
+ 已有 observation.state 时不动(尊重上游的显式声明)。返回派生列名或
1151
+ None。
1152
+ """
1153
+ import numpy as np
1154
+
1155
+ if any("observation.state" in row for row in rows):
1156
+ return None
1157
+ if not any("observation.slam_pose" in row for row in rows):
1158
+ return None
1159
+
1160
+ # 按 episode_index 分组:相对偏移必须在**各自集内**计算,跨集求差
1161
+ # 会把两集世界原点的差异当成运动。
1162
+ by_episode: dict[int, list[dict]] = {}
1163
+ for row in rows:
1164
+ try:
1165
+ episode = int(row.get("episode_index", 0))
1166
+ except (TypeError, ValueError):
1167
+ episode = 0
1168
+ by_episode.setdefault(episode, []).append(row)
1169
+
1170
+ derived = 0
1171
+ for episode_rows in by_episode.values():
1172
+ origin = None
1173
+ for row in episode_rows:
1174
+ pose = row.get("observation.slam_pose")
1175
+ try:
1176
+ pose_values = np.asarray(pose, dtype=float).reshape(-1)
1177
+ except (TypeError, ValueError):
1178
+ continue
1179
+ if pose_values.size < 3 or not np.isfinite(pose_values[:3]).all():
1180
+ continue
1181
+ if origin is None:
1182
+ origin = pose_values[:3].copy()
1183
+ gripper = 0.0
1184
+ raw = row.get("observation.gripper_state")
1185
+ try:
1186
+ gripper_values = np.asarray(raw, dtype=float).reshape(-1)
1187
+ if gripper_values.size:
1188
+ gripper = float(np.clip(gripper_values[0] / 100.0, 0.0, 1.0))
1189
+ except (TypeError, ValueError):
1190
+ pass
1191
+ row["observation.state"] = [
1192
+ float(value) for value in np.concatenate([
1193
+ pose_values[:3] - origin, [gripper]])
1194
+ ]
1195
+ derived += 1
1196
+ if not derived:
1197
+ return None
1198
+ print(f"[lerobot_export] derived observation.state for {derived} frames")
1199
+ return "observation.state"
1200
+
1201
+
1202
+ def _ensure_slam_pose(rows: list[dict]) -> str | None:
1203
+ """确保每帧有 ``observation.slam_pose``;缺失时由 ``slam_trajectory`` 重建。
1204
+
1205
+ 采集端从 52 集起不再写 ``observation.slam_pose``,只写高频样本缓冲
1206
+ ``observation.slam_trajectory``(旧批两列都有)。ACT 的输入特征里声明了
1207
+ slam_pose,整列缺失会让官方加载器与策略直接 KeyError —— 这批数据因此
1208
+ 完全无法训练或评估。
1209
+
1210
+ 重建口径见 ``app.umi_slam_action.poses_from_trajectory``(与派生 action
1211
+ 用的是同一个函数)。拿旧集(有 slam_pose)做过对照:位置相关系数
1212
+ 0.998~0.9999、逐帧位置差 std 仅 1~4mm(亚帧采样差),数值范围完全一致,
1213
+ 四元数 |dot| 中位 1.0000 —— **同一个物理量、同一个世界系**,不是近似。
1214
+ 重建版反而补上了采集端留的 28.5% 零填充行,质量更好。
1215
+
1216
+ 起点偏移的口径与 ``_ensure_observation_state`` 一致,故两者可以串联:本
1217
+ 函数先跑,缺 state 的集还能继续从重建出的 slam_pose 派生 state。
1218
+
1219
+ 已有该列时不动(旧批重导行为完全不变)。返回写入的列名或 None。
1220
+ """
1221
+ import numpy as np
1222
+
1223
+ if any("observation.slam_pose" in row for row in rows):
1224
+ return None
1225
+ if not any("observation.slam_trajectory" in row for row in rows):
1226
+ return None
1227
+
1228
+ from app.umi_slam_action import poses_from_trajectory
1229
+
1230
+ # 按 episode_index 分组:时间基映射是每集单独拟合的(各集 SLAM 会话
1231
+ # 独立),跨集混在一起会把不同集的时间基搅乱,重建出的位姿全错。
1232
+ by_episode: dict[int, list[dict]] = {}
1233
+ for row in rows:
1234
+ try:
1235
+ episode = int(row.get("episode_index", 0))
1236
+ except (TypeError, ValueError):
1237
+ episode = 0
1238
+ by_episode.setdefault(episode, []).append(row)
1239
+
1240
+ written = 0
1241
+ for episode_rows in by_episode.values():
1242
+ count = len(episode_rows)
1243
+ trajectories = [row.get("observation.slam_trajectory")
1244
+ for row in episode_rows]
1245
+ # 时间戳必须是稠密 float 数组:缺帧补 NaN(poses_from_trajectory 会
1246
+ # 把它们标为缺失再插值),混进 None 会让 np.asarray 推断出 object。
1247
+ stamps = np.full(count, np.nan, dtype=float)
1248
+ for index, row in enumerate(episode_rows):
1249
+ try:
1250
+ stamps[index] = float(row.get("timestamp"))
1251
+ except (TypeError, ValueError):
1252
+ continue
1253
+ poses = poses_from_trajectory(trajectories, stamps, count)
1254
+ if poses is None:
1255
+ continue
1256
+ for row, pose in zip(episode_rows, poses):
1257
+ row["observation.slam_pose"] = [float(value) for value in pose]
1258
+ written += 1
1259
+ if not written:
1260
+ return None
1261
+ print(f"[lerobot_export] derived observation.slam_pose for {written} frames")
1262
+ return "observation.slam_pose"
1263
+
1264
+
1265
+ def _pad_sparse_matrix_columns(rows: list[dict], nan_fill: float) -> dict[str, int]:
1266
+ """定长矩阵/trajectory 列的空帧补齐到声明长度。
1267
+
1268
+ 采集端在尚无数据的帧上写空数组(如触觉矩阵首帧、SLAM 丢帧时的
1269
+ trajectory),同一列因此长短不一。官方加载器按声明的 shape 转成定长
1270
+ Sequence,遇到空值直接 cast 失败:
1271
+ TypeError: Couldn't cast array of type list<element: int64>
1272
+ to List(Value('int64'), length=187500)
1273
+
1274
+ 只补空值,不动有值帧 —— 缺失语义由列本身的值域表达。
1275
+
1276
+ 返回 ``{列名: 补齐后的长度}``:features 声明必须用这个长度,而不是
1277
+ 首帧样本的长度 —— 首帧常常正是空的那一帧,拿它声明会得到 shape [0],
1278
+ 与实际写入的长度不符,加载器照样 cast 失败。
1279
+ """
1280
+ import numpy as np
1281
+ targets: dict[str, int] = {}
1282
+ for key in {k for row in rows for k in row}:
1283
+ if not (key.endswith("_matrix") or key.endswith("_trajectory")):
1284
+ continue
1285
+ target = 0
1286
+ sample_value = None
1287
+ for row in rows:
1288
+ value = row.get(key)
1289
+ if isinstance(value, (list, tuple)) and len(value) > target:
1290
+ target = len(value)
1291
+ sample_value = value
1292
+ if not target or not sample_value:
1293
+ continue
1294
+ targets[key] = target
1295
+ # 补齐值的类型必须与该列已有元素**逐位一致**:触觉矩阵是 int16
1296
+ # (声明 int64),补浮点会让 pyarrow 把整列推断成 double,与声明的
1297
+ # dtype 冲突 → 加载器 cast 失败。int 列补 0,浮点列补 nan_fill。
1298
+ existing = next((v for v in sample_value if v is not None), None)
1299
+ if isinstance(existing, (int, np.integer)) and not isinstance(existing, bool):
1300
+ fill = 0
1301
+ else:
1302
+ fill = nan_fill
1303
+ for row in rows:
1304
+ value = row.get(key)
1305
+ if not isinstance(value, (list, tuple)) or len(value) == target:
1306
+ continue
1307
+ row[key] = list(value) + [fill] * (target - len(value))
1308
+ return targets
1309
+
1310
+
1311
  def _pad_variable_columns(rows: list[dict],
1312
+ nan_fill: float = float("nan")) -> tuple[int, dict[str, int]]:
1313
+ """变长/稀疏列固定长度化。返回 ``(imu_cap, {列名: 补齐长度})``。
 
1314
 
1315
  - observation.imu:每样本 6 维 → 展平为 cap*6(nan_fill 补尾)
1316
  - observation.imu_ts_ns:补 0 到 cap
1317
+ - 矩阵/trajectory 类的空帧:补齐到该列最大长度
1318
+ (见 _pad_sparse_matrix_columns)
1319
+
1320
+ 第二个返回值供 features 声明使用 —— 声明必须等于**实际写入的长度**,
1321
+ 否则官方加载器 cast 失败。
1322
  """
1323
  import numpy as np
1324
+ matrix_targets = _pad_sparse_matrix_columns(rows, nan_fill)
1325
  imu_cap = 3
1326
  for row in rows:
1327
  for key in ("observation.imu", "observation.imu_ts_ns"):
 
1329
  if isinstance(value, (list, tuple)):
1330
  imu_cap = max(imu_cap, len(value))
1331
  if not any("observation.imu" in row for row in rows):
1332
+ return imu_cap, matrix_targets
1333
  for row in rows:
1334
  imu = row.get("observation.imu")
1335
  if isinstance(imu, (list, tuple)):
 
1353
  if isinstance(ts, (list, tuple)):
1354
  row["observation.imu_ts_ns"] = (
1355
  [int(v) for v in ts] + [0] * (imu_cap - len(ts)))[:imu_cap]
1356
+ return imu_cap, matrix_targets
1357
 
1358
 
1359
  def _propagated_hand_sets(source_row: dict) -> tuple[set[str], set[str], dict[str, set[str]]]:
 
1915
  hand_3d_right_paths: list[str] | None = None,
1916
  version: str = "v3.0",
1917
  hand_3d_unit: str | None = None,
1918
+ exclude_columns: list[str] | None = None,
1919
  progress_callback=None) -> Path:
1920
  """Build a LeRobot dataset (``version`` = "v2.1" | "v3.0").
1921
 
1922
+ ``exclude_columns``: 可选,精确列名列表,从数据集中整体剔除。
1923
+ 用于丢掉训练用不到但体积/计算代价极高的列(如 250×250×3 的触觉
1924
+ 力矩阵:每帧 0.8MB,且逐元素分位数统计是导出耗时的主要来源)。
1925
+ 被剔除的列不会出现在 parquet、features 声明和 stats 里。
1926
+
1927
  Depth sources are exported as their preview video (``is_depth_map``)
1928
  plus the per-frame ``observation.depth.<source>.valid`` marker; the
1929
  original 16-bit PNG sequence stays in the session directory and is not
 
2008
  # 本集实际导出的每路视频(帧数, fps) → episodes parquet 的
2009
  # videos/<key>/from_timestamp 映射(官方 v3 加载器按时间戳定位帧)
2010
  episode_video_meta: dict[str, tuple[int, float]] = {}
2011
+ sensor_rows = _read_sensor_rows(
2012
+ session_dir, episode_id,
2013
+ skip_columns=set(str(name) for name in exclude_columns or []))
2014
  annotation_map, annotation_defs, _skipped_candidates = _annotation_maps(episode_id)
2015
 
2016
  description = str(episode.get("project") or "").strip()
 
2052
  / f"observation.images.{source_key}")
2053
  output_video = destination / f"file-{out_episode_index:03d}.mp4"
2054
  destination.mkdir(parents=True, exist_ok=True)
2055
+ _write_video_dense_keyframes(source, output_video)
2056
  total_videos += 1
2057
  probed_count, probed_fps, width, height = _probe_video(source)
2058
  episode_video_meta[f"observation.images.{source_key}"] = (
 
2286
  except Exception:
2287
  pass # 进度回调失败不影响导出本身
2288
 
2289
+ # 调用方指定剔除的列(如触觉力矩阵):在补齐/统计/写盘之前删掉,
2290
+ # 这样 parquet、features 声明与 stats 三处天然一致,不需要各自过滤。
2291
+ if exclude_columns:
2292
+ dropped = set(str(name) for name in exclude_columns)
2293
+ if dropped:
2294
+ for row in rows:
2295
+ for key in dropped:
2296
+ row.pop(key, None)
2297
+ print(f"[lerobot_export] excluded columns: {sorted(dropped)}")
2298
+
2299
+ # SLAM 位姿列:采集端 52 集起只写 slam_trajectory,不再写 slam_pose。
2300
+ # 必须赶在 _pad_variable_columns 之前做 —— 补齐会把冲空帧填成 0,而
2301
+ # 0 在 slam_trajectory 里是一个"合法样本"(t=0 的位姿),重建时会被
2302
+ # 当成真实帧混进去。此处补出的 slam_pose 又是下面 observation.state
2303
+ # 的派生依据,故顺序不能颠倒。
2304
+ _ensure_slam_pose(rows)
2305
+
2306
+ # ACT 的观测需要 proprioception:单帧图像不足以让模型知道夹爪当前
2307
+ # 在哪。数据集若已有 observation.state 则原样保留;否则在 UMI 会话上
2308
+ # 由 SLAM 位姿 + 夹爪开合派生 [Δx, Δy, Δz, gripper](位置为相对本集
2309
+ # 起点的偏移 —— SLAM 世界原点是每集任意初始化的,绝对坐标是噪声)。
2310
+ _ensure_observation_state(rows)
2311
+
2312
  # 变长异步传感器列固定长度化:官方加载器不支持变长列(实测
2313
  # list<int64> 对声明 int64[1] 直接 cast 失败)。取全数据集最大
2314
  # 样本数补齐,声明 shape 同步为固定长度,变长语义保留在 note。
2315
+ imu_cap, padded_lengths = _pad_variable_columns(
2316
  rows, nan_fill=0.0 if is_v2 else float("nan"))
2317
 
2318
  # 收敛:删恒定哨兵垃圾列(全程 NaN/-1/0/空串 → 数据从未写入)。
 
2585
  import numpy as np
2586
  arr = np.asarray(value)
2587
  shape = list(arr.shape) if arr.ndim else [1]
2588
+ # 补齐过的列必须以实际写入的长度声明。首帧样本往往正是空的那
2589
+ # 一帧(触觉矩阵首帧、SLAM 丢帧),照它声明会得到 shape [0] 或
2590
+ # 偏小的长度,与实际写入的数据不符 → 官方加载器 cast 失败。
2591
+ if key in padded_lengths:
2592
+ shape = [padded_lengths[key]]
2593
  if key in ("observation.tactile.left", "observation.tactile.right"):
2594
  features[key] = {
2595
  "dtype": "float32", "shape": [256],
 
2630
  "shape": shape}
2631
  if junk_cols:
2632
  features = {k: v for k, v in features.items() if k not in junk_cols}
2633
+ # 调用方剔除的列同样从声明里去掉。features 由 sensor_rows 样本推导,
2634
+ # 而剔除只作用于 rows,不过滤这里会留下"声明了但数据里没有"的列,
2635
+ # 官方加载器读不出来会报错。
2636
+ if exclude_columns:
2637
+ excluded = set(str(name) for name in exclude_columns)
2638
+ features = {k: v for k, v in features.items() if k not in excluded}
2639
+ # 导出的行里有 observation.state 就必须声明,否则官方加载器读不完整。
2640
+ # 声明放在剔除之后:调用方显式排除的列不该被这里复活。
2641
+ # 补出来的 slam_pose 同样必须声明:features 是从**源 parquet** 采样的
2642
+ # (sensor_rows),而这列由 _ensure_slam_pose 在内存里写进 rows,采样
2643
+ # 看不到它 —— 不补声明就是"数据里有列、声明里没有",官方加载器读不完整。
2644
+ if rows and "observation.slam_pose" in rows[0] and "observation.slam_pose" not in features:
2645
+ features["observation.slam_pose"] = {
2646
+ "dtype": "float32", "shape": [7],
2647
+ "names": ["x", "y", "z", "qx", "qy", "qz", "qw"],
2648
+ "note": ("世界系绝对位姿(米 + 四元数 xyzw)。采集端 52 集起不再"
2649
+ "写入该列,导出时由 observation.slam_trajectory 的 50Hz"
2650
+ "样本重建(与采集端原始列实测位置相关 0.998+,同一世界系)。"),
2651
+ }
2652
+ if rows and "observation.state" in rows[0] and "observation.state" not in features:
2653
+ features["observation.state"] = {
2654
+ "dtype": "float32", "shape": [4],
2655
+ "names": ["delta_x", "delta_y", "delta_z", "gripper"],
2656
+ "note": ("proprioception:本帧位置相对**本集第一帧**的偏移(米)"
2657
+ "+ 夹爪开合(0–1)。SLAM 世界原点是每集任意初始化的,"
2658
+ "故用相对量而非绝对坐标。"),
2659
+ }
2660
  # 深度有效帧标记列(行循环写入,不声明会导致官方加载器读不完整)
2661
  for depth_key in sorted(depth_source_keys):
2662
  key = f"observation.depth.{depth_key}.valid"
processor/app/localstore.py CHANGED
@@ -37,6 +37,7 @@ _SESSIONS_CACHE_TTL = 300.0
37
  # 全量遍历远程挂载很慢:只允许单一线程执行(_scan_lock),遍历本身不持有
38
  # _lock —— 缓存过期触发的重扫不再把状态读写请求拖在锁后面排队。
39
  _scan_lock = threading.Lock()
 
40
  _scan_generation = 0 # invalidate_session_cache 递增;并发扫描据此丢弃过期结果
41
  _state_write_generation = 0 # write_episode_state 递增;扫描期间的状态写入触发重扫
42
  _video_probe_cache: dict[str, tuple[int, float]] = {}
@@ -48,6 +49,14 @@ _workflows_cache: list[dict] | None = None
48
  _workflows_cache_at = 0.0
49
  _runs_cache: list[dict] | None = None
50
  _runs_cache_at = 0.0
 
 
 
 
 
 
 
 
51
  # Runs are written by the API process and read by separate worker processes.
52
  # A ten-minute process-local cache can hide a newly queued upload/reprocess
53
  # from workers. Keep the cache short while still coalescing rapid UI scans.
@@ -129,6 +138,14 @@ def invalidate_session_cache() -> None:
129
  _sessions_cache = None
130
  _sessions_cache_at = 0.0
131
  _video_probe_cache.clear()
 
 
 
 
 
 
 
 
132
 
133
 
134
  def scan_sessions() -> list[dict]:
@@ -141,8 +158,53 @@ def scan_sessions() -> list[dict]:
141
  _lock:缓存过期触发的重扫不再把状态读写等无关请求拖在锁后面排队。
142
  遍历期间发生结构变化(invalidate)/状态写入时,按代际计数丢弃过期结果
143
  重扫一次。
 
 
 
 
 
144
  """
145
  global _sessions_cache, _sessions_cache_at, _task_sessions_cache
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
146
  while True:
147
  with _lock:
148
  now = time.monotonic()
@@ -210,11 +272,19 @@ def _build_sessions_snapshot() -> list[dict]:
210
  def _scan_project_dir(project_dir: Path, project: str) -> list[dict]:
211
  """Scan a project-level LeRobot tree and expose one row per episode."""
212
  from app.lerobot_v21 import is_depth_source
213
- from app.project_dataset import episode_files, project_episode_rows
 
 
 
 
214
 
215
  info = _read_json(project_dir / "meta" / "info.json", {})
216
  if not isinstance(info, dict):
217
  info = {}
 
 
 
 
218
  rows: list[dict] = []
219
  for row in project_episode_rows(project_dir):
220
  try:
@@ -223,7 +293,7 @@ def _scan_project_dir(project_dir: Path, project: str) -> list[dict]:
223
  continue
224
  episode_id = str(row.get("episode_id") or row.get("source_batch")
225
  or f"{project}_{episode_index:06d}")
226
- files = episode_files(project_dir, episode_index)
227
  state = read_episode_state(episode_id)
228
  # Project-level info is an aggregate. Do not let it make a new
229
  # one-camera episode display devices that belong to another episode.
@@ -683,6 +753,13 @@ def delete_episode(episode_id: str, permanent: bool = False,
683
  _remove_json(EPISODE_STATES_DIR / f"{name}.json")
684
  _remove_json(ANNOTATIONS_DIR / f"{name}.json")
685
  _episode_state_cache.pop(name, None)
 
 
 
 
 
 
 
686
  global _deleted_episodes_cache, _deleted_episodes_cache_at
687
  _deleted_episodes_cache = None
688
  _deleted_episodes_cache_at = 0.0
@@ -1190,38 +1267,198 @@ def mutate_annotation_by_id(
1190
  # ── 工作流运行队列 ───────────────────────────────────
1191
 
1192
  RUNS_DIR = STATE_ROOT / "runs"
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1193
 
 
 
 
1194
 
1195
- def list_runs() -> list[dict]:
1196
- global _runs_cache, _runs_cache_at
 
 
 
 
 
 
 
 
1197
  now = time.monotonic()
1198
- if (_runs_cache is not None
1199
- and now - _runs_cache_at < _STATE_LIST_CACHE_TTL):
1200
- return copy.deepcopy(_runs_cache)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1201
  if not RUNS_DIR.is_dir():
1202
- _runs_cache = []
1203
- _runs_cache_at = now
1204
- return []
1205
- runs = []
1206
  for f in sorted(RUNS_DIR.glob("*.json")):
1207
  run = _read_json(f, None)
1208
- if isinstance(run, dict):
1209
- runs.append(run)
1210
- _runs_cache = runs
1211
- _runs_cache_at = time.monotonic()
1212
- return copy.deepcopy(runs)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1213
 
1214
 
1215
  def get_run(run_id: str) -> dict | None:
1216
- return _read_json(RUNS_DIR / f"{run_id}.json", None)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1217
 
1218
 
1219
  def save_run(run: dict) -> None:
1220
- global _runs_cache, _runs_cache_at
1221
  with _lock:
1222
- _write_json(RUNS_DIR / f"{run.get('id')}.json", run)
1223
- _runs_cache = None
1224
- _runs_cache_at = 0.0
1225
 
1226
 
1227
  def update_run_if_owned(
@@ -1231,7 +1468,6 @@ def update_run_if_owned(
1231
  mutator: Callable[[dict], None],
1232
  ) -> dict | None:
1233
  """Atomically mutate a running job only for its current lease owner."""
1234
- global _runs_cache, _runs_cache_at
1235
  with _lock:
1236
  path = RUNS_DIR / f"{run_id}.json"
1237
  run = _read_json(path, None)
@@ -1242,9 +1478,10 @@ def update_run_if_owned(
1242
  or run.get("lease_token") != lease_token):
1243
  return None
1244
  mutator(run)
1245
- _write_json(path, run)
1246
- _runs_cache = None
1247
- _runs_cache_at = 0.0
 
1248
  return run
1249
 
1250
 
@@ -1265,7 +1502,6 @@ def save_run_if_absent(
1265
  ``workflow_revision`` 时,queued/running 仍视为占用,completed 则
1266
  允许第一次迁移到带版本的幂等记录。
1267
  """
1268
- global _runs_cache, _runs_cache_at
1269
  with _lock:
1270
  revision = run.get("workflow_revision")
1271
  for existing in list_runs():
@@ -1280,23 +1516,21 @@ def save_run_if_absent(
1280
  existing["status"] = "superseded"
1281
  existing["finished_at"] = utcnow_iso()
1282
  existing["error_log"] = "Superseded by a newer upload or reprocess request"
1283
- _write_json(RUNS_DIR / f"{existing.get('id')}.json", existing)
1284
- _runs_cache = None
1285
- _runs_cache_at = 0.0
1286
  continue
1287
  if same_revision or legacy_active:
1288
  if existing.get("status") in ("queued", "running"):
1289
  return existing, False
1290
  if existing.get("status") == "completed" and not allow_completed_rerun:
1291
  return existing, False
1292
- _write_json(RUNS_DIR / f"{run.get('id')}.json", run)
1293
- _runs_cache = None
1294
- _runs_cache_at = 0.0
1295
  return run, True
1296
 
1297
 
1298
  def delete_run(run_id: str) -> None:
1299
- global _runs_cache, _runs_cache_at
1300
  _remove_json(RUNS_DIR / f"{run_id}.json")
1301
- _runs_cache = None
1302
- _runs_cache_at = 0.0
 
37
  # 全量遍历远程挂载很慢:只允许单一线程执行(_scan_lock),遍历本身不持有
38
  # _lock —— 缓存过期触发的重扫不再把状态读写请求拖在锁后面排队。
39
  _scan_lock = threading.Lock()
40
+ _sessions_refresh_running = False # 后台 stale-while-revalidate 单飞标志
41
  _scan_generation = 0 # invalidate_session_cache 递增;并发扫描据此丢弃过期结果
42
  _state_write_generation = 0 # write_episode_state 递增;扫描期间的状态写入触发重扫
43
  _video_probe_cache: dict[str, tuple[int, float]] = {}
 
49
  _workflows_cache_at = 0.0
50
  _runs_cache: list[dict] | None = None
51
  _runs_cache_at = 0.0
52
+ _runs_cache_revision: tuple = ()
53
+ # list_runs(include_archived=False) keeps its own slot; the two variants are
54
+ # cached independently so neither call pattern can serve the other's result.
55
+ _runs_live_cache: list[dict] | None = None
56
+ _runs_live_cache_at = 0.0
57
+ _runs_live_cache_revision: tuple = ()
58
+ # 指纹变化时才重建快照;单飞,避免并发请求各读一遍 539 个 JSON。
59
+ _runs_scan_lock = threading.Lock()
60
  # Runs are written by the API process and read by separate worker processes.
61
  # A ten-minute process-local cache can hide a newly queued upload/reprocess
62
  # from workers. Keep the cache short while still coalescing rapid UI scans.
 
138
  _sessions_cache = None
139
  _sessions_cache_at = 0.0
140
  _video_probe_cache.clear()
141
+ # 项目索引缓存(project_dataset)按指纹自校验,但上传/删除/改名必须
142
+ # 立刻可见,不能等 TTL。懒导入避免模块级循环依赖。
143
+ try:
144
+ from app.project_dataset import invalidate_project_cache
145
+
146
+ invalidate_project_cache()
147
+ except Exception as exc:
148
+ print(f"[Cache] Project index invalidation skipped: {exc}")
149
 
150
 
151
  def scan_sessions() -> list[dict]:
 
158
  _lock:缓存过期触发的重扫不再把状态读写等无关请求拖在锁后面排队。
159
  遍历期间发生结构变化(invalidate)/状态写入时,按代际计数丢弃过期结果
160
  重扫一次。
161
+
162
+ TTL 过期但已有快照时走 stale-while-revalidate:立刻返回旧快照,后台单飞
163
+ 重建。全量遍历要 5s+,让审核页的请求撞上它正是"打开一条数据等十几秒"
164
+ 的一个来源;旧快照里缺的只是刚上传的那一集,而上传会走
165
+ invalidate_session_cache() 清空缓存,所以这里不会藏住新数据。
166
  """
167
  global _sessions_cache, _sessions_cache_at, _task_sessions_cache
168
+ with _lock:
169
+ snapshot = (copy.deepcopy(_sessions_cache)
170
+ if _sessions_cache is not None else None)
171
+ fresh = (snapshot is not None
172
+ and time.monotonic() - _sessions_cache_at < _SESSIONS_CACHE_TTL)
173
+ if fresh:
174
+ return snapshot
175
+ if snapshot is not None:
176
+ _refresh_sessions_in_background()
177
+ return snapshot
178
+ # 冷启动或刚被 invalidate:必须同步扫出第一份快照。
179
+ return _scan_sessions_blocking()
180
+
181
+
182
+ def _refresh_sessions_in_background() -> None:
183
+ """Single-flight background rebuild of the expired session snapshot."""
184
+ global _sessions_refresh_running
185
+ with _lock:
186
+ if _sessions_refresh_running:
187
+ return
188
+ _sessions_refresh_running = True
189
+ threading.Thread(
190
+ target=_run_session_refresh, name="session-refresh", daemon=True
191
+ ).start()
192
+
193
+
194
+ def _run_session_refresh() -> None:
195
+ global _sessions_refresh_running
196
+ try:
197
+ _scan_sessions_blocking()
198
+ except Exception as exc:
199
+ print(f"[Cache] Background session refresh failed: {exc}")
200
+ finally:
201
+ with _lock:
202
+ _sessions_refresh_running = False
203
+
204
+
205
+ def _scan_sessions_blocking() -> list[dict]:
206
+ """Full remote tree walk; callers block for the duration."""
207
+ global _sessions_cache, _sessions_cache_at, _task_sessions_cache
208
  while True:
209
  with _lock:
210
  now = time.monotonic()
 
272
  def _scan_project_dir(project_dir: Path, project: str) -> list[dict]:
273
  """Scan a project-level LeRobot tree and expose one row per episode."""
274
  from app.lerobot_v21 import is_depth_source
275
+ from app.project_dataset import (
276
+ episode_files_from_index,
277
+ project_episode_file_index,
278
+ project_episode_rows,
279
+ )
280
 
281
  info = _read_json(project_dir / "meta" / "info.json", {})
282
  if not isinstance(info, dict):
283
  info = {}
284
+ # One index for the whole project: asking ``episode_files`` per episode
285
+ # would re-walk data/, videos/ and every episode index file each time,
286
+ # costing (1 + episodes) x parquet reads on the storage mount.
287
+ file_index = project_episode_file_index(project_dir)
288
  rows: list[dict] = []
289
  for row in project_episode_rows(project_dir):
290
  try:
 
293
  continue
294
  episode_id = str(row.get("episode_id") or row.get("source_batch")
295
  or f"{project}_{episode_index:06d}")
296
+ files = episode_files_from_index(project_dir, episode_index, file_index)
297
  state = read_episode_state(episode_id)
298
  # Project-level info is an aggregate. Do not let it make a new
299
  # one-camera episode display devices that belong to another episode.
 
753
  _remove_json(EPISODE_STATES_DIR / f"{name}.json")
754
  _remove_json(ANNOTATIONS_DIR / f"{name}.json")
755
  _episode_state_cache.pop(name, None)
756
+ # 内容指纹指向已删除的批次:清掉,否则同一条录制重新上传时会被
757
+ # 误判成"重复"而试图覆盖一个已经不存在的 episode。
758
+ try:
759
+ from app.batch_fingerprint import forget_fingerprint
760
+ forget_fingerprint(name)
761
+ except Exception:
762
+ pass
763
  global _deleted_episodes_cache, _deleted_episodes_cache_at
764
  _deleted_episodes_cache = None
765
  _deleted_episodes_cache_at = 0.0
 
1267
  # ── 工作流运行队列 ───────────────────────────────────
1268
 
1269
  RUNS_DIR = STATE_ROOT / "runs"
1270
+ # Finished runs keep their full record for the UI, but they are moved out of
1271
+ # the live queue directory: the worker polls /jobs/claim every few seconds and
1272
+ # used to re-read every historical run (hundreds of 26 KB JSON files on the
1273
+ # NAS mount) on each poll. Only unfinished work stays in RUNS_DIR.
1274
+ RUNS_ARCHIVE_DIR = RUNS_DIR / "archive"
1275
+ _TERMINAL_RUN_STATUSES = frozenset({"completed", "failed", "superseded"})
1276
+
1277
+ # claim_job only needs these to pick the next job; the full run payload is
1278
+ # loaded afterwards for the winner alone.
1279
+ _RUN_SUMMARY_FIELDS = ("id", "status", "created_at", "lease_until",
1280
+ "worker_id", "lease_token", "attempt")
1281
+
1282
+
1283
+ def list_runs(include_archived: bool = True) -> list[dict]:
1284
+ """All runs, newest-finished records included by default.
1285
 
1286
+ ``include_archived=False`` restricts the scan to the live queue and is
1287
+ only for callers that just need claimable work. Everything else keeps
1288
+ the historical view it had before archiving existed.
1289
 
1290
+ The two variants are cached separately — a single shared cache would
1291
+ serve whichever variant ran first to both callers.
1292
+
1293
+ TTL 过期后先比一次廉价指纹(目录 listing + stat,539 个 run 实测 4ms),
1294
+ 内容没变就只刷新时间戳。历史快照要读 539 个 JSON,冷读 6s —— 那正是
1295
+ ``/export/episode-format`` 每次点开数据都要付的代价;而 worker 领取
1296
+ 走的是 :func:`list_run_summaries`,不经过这里,所以缓存不会耽误派活。
1297
+ """
1298
+ global _runs_cache, _runs_cache_at, _runs_cache_revision
1299
+ global _runs_live_cache, _runs_live_cache_at, _runs_live_cache_revision
1300
  now = time.monotonic()
1301
+ cache, cache_at = ((_runs_cache, _runs_cache_at) if include_archived
1302
+ else (_runs_live_cache, _runs_live_cache_at))
1303
+ if cache is not None and now - cache_at < _STATE_LIST_CACHE_TTL:
1304
+ return copy.deepcopy(cache)
1305
+ revision = _run_dirs_revision(include_archived)
1306
+ cached_revision = (_runs_cache_revision if include_archived
1307
+ else _runs_live_cache_revision)
1308
+ if cache is not None and cached_revision == revision:
1309
+ if include_archived:
1310
+ _runs_cache_at = now
1311
+ else:
1312
+ _runs_live_cache_at = now
1313
+ return copy.deepcopy(cache)
1314
+ # 指纹变化时才真读;单飞避免并发请求各读一遍 539 个文件。
1315
+ with _runs_scan_lock:
1316
+ now = time.monotonic()
1317
+ cache, cache_at = ((_runs_cache, _runs_cache_at) if include_archived
1318
+ else (_runs_live_cache, _runs_live_cache_at))
1319
+ cached_revision = (_runs_cache_revision if include_archived
1320
+ else _runs_live_cache_revision)
1321
+ if cache is not None and (cached_revision == revision
1322
+ or now - cache_at < _STATE_LIST_CACHE_TTL):
1323
+ return copy.deepcopy(cache)
1324
+ runs: list[dict] = []
1325
+ for directory in _run_dirs(include_archived):
1326
+ if not directory.is_dir():
1327
+ continue
1328
+ for f in sorted(directory.glob("*.json")):
1329
+ run = _read_json(f, None)
1330
+ if isinstance(run, dict):
1331
+ runs.append(run)
1332
+ revision = _run_dirs_revision(include_archived)
1333
+ if include_archived:
1334
+ _runs_cache = runs
1335
+ _runs_cache_at = time.monotonic()
1336
+ _runs_cache_revision = revision
1337
+ else:
1338
+ _runs_live_cache = runs
1339
+ _runs_live_cache_at = time.monotonic()
1340
+ _runs_live_cache_revision = revision
1341
+ return copy.deepcopy(runs)
1342
+
1343
+
1344
+ def _run_dirs_revision(include_archived: bool) -> tuple:
1345
+ """Cheap fingerprint of the run files (name, size, mtime)."""
1346
+ parts: list[tuple] = []
1347
+ try:
1348
+ for directory in _run_dirs(include_archived):
1349
+ if not directory.is_dir():
1350
+ continue
1351
+ for f in sorted(directory.glob("*.json")):
1352
+ stat = f.stat()
1353
+ parts.append((f.name, stat.st_size, stat.st_mtime_ns))
1354
+ except OSError:
1355
+ return ()
1356
+ return tuple(parts)
1357
+
1358
+
1359
+ def _run_dirs(include_archived: bool = False) -> list[Path]:
1360
+ """Live queue directory first; the archive only when explicitly asked."""
1361
+ dirs = [RUNS_DIR]
1362
+ if include_archived:
1363
+ dirs.append(RUNS_ARCHIVE_DIR)
1364
+ return dirs
1365
+
1366
+
1367
+ def _invalidate_runs_cache() -> None:
1368
+ """Drop both run-list caches. Callers must hold ``_lock``."""
1369
+ global _runs_cache, _runs_cache_at, _runs_live_cache, _runs_live_cache_at
1370
+ _runs_cache = None
1371
+ _runs_cache_at = 0.0
1372
+ _runs_live_cache = None
1373
+ _runs_live_cache_at = 0.0
1374
+
1375
+
1376
+ def list_run_summaries() -> list[dict]:
1377
+ """Live runs with only the fields the claim scan needs.
1378
+
1379
+ Parsing 26 KB of JSON per historical run is what made an idle worker burn
1380
+ the backend's CPU; the claim path only compares ``status`` /
1381
+ ``created_at`` / ``lease_until``. Archived (terminal) runs are skipped
1382
+ entirely — nothing in the live queue can be claimed from them.
1383
+ """
1384
+ summaries: list[dict] = []
1385
  if not RUNS_DIR.is_dir():
1386
+ return summaries
 
 
 
1387
  for f in sorted(RUNS_DIR.glob("*.json")):
1388
  run = _read_json(f, None)
1389
+ if not isinstance(run, dict):
1390
+ continue
1391
+ summaries.append({key: run.get(key) for key in _RUN_SUMMARY_FIELDS})
1392
+ return summaries
1393
+
1394
+
1395
+ def archive_run_file(run_id: str) -> bool:
1396
+ """Move a finished run out of the live queue directory.
1397
+
1398
+ Keeps the record (the UI can still load it by id) while taking it out of
1399
+ the per-poll directory listing. Safe to call repeatedly.
1400
+ """
1401
+ source = RUNS_DIR / f"{run_id}.json"
1402
+ if not source.is_file():
1403
+ return False
1404
+ try:
1405
+ RUNS_ARCHIVE_DIR.mkdir(parents=True, exist_ok=True)
1406
+ destination = RUNS_ARCHIVE_DIR / source.name
1407
+ if destination.exists():
1408
+ destination.unlink()
1409
+ source.replace(destination)
1410
+ except OSError:
1411
+ return False
1412
+ with _lock:
1413
+ _invalidate_runs_cache()
1414
+ return True
1415
 
1416
 
1417
  def get_run(run_id: str) -> dict | None:
1418
+ """Load one run by id from the live queue, falling back to the archive."""
1419
+ run = _read_json(RUNS_DIR / f"{run_id}.json", None)
1420
+ if isinstance(run, dict):
1421
+ return run
1422
+ return _read_json(RUNS_ARCHIVE_DIR / f"{run_id}.json", None)
1423
+
1424
+
1425
+ def _write_run_record(run: dict) -> None:
1426
+ """Persist a run, keeping finished ones out of the live queue directory.
1427
+
1428
+ Callers must hold ``_lock``. Terminal runs are archived so every
1429
+ completion path (worker complete/fail, dispatch, API status writes, the
1430
+ supersede branch) stops contributing to the claim scan without each site
1431
+ having to remember. A run that leaves a terminal state is pulled back
1432
+ into the live queue.
1433
+ """
1434
+ run_id = str(run.get("id") or "")
1435
+ _write_json(RUNS_DIR / f"{run_id}.json", run)
1436
+ if not run_id:
1437
+ return
1438
+ if run.get("status") in _TERMINAL_RUN_STATUSES:
1439
+ try:
1440
+ RUNS_ARCHIVE_DIR.mkdir(parents=True, exist_ok=True)
1441
+ live = RUNS_DIR / f"{run_id}.json"
1442
+ if live.is_file():
1443
+ live.replace(RUNS_ARCHIVE_DIR / f"{run_id}.json")
1444
+ except OSError:
1445
+ pass
1446
+ else:
1447
+ # Re-queued after a re-run: drop the stale archived copy so the run
1448
+ # is not listed twice.
1449
+ stale = RUNS_ARCHIVE_DIR / f"{run_id}.json"
1450
+ try:
1451
+ if stale.is_file():
1452
+ stale.unlink()
1453
+ except OSError:
1454
+ pass
1455
 
1456
 
1457
  def save_run(run: dict) -> None:
1458
+ """Write a run, keeping finished runs out of the live queue directory."""
1459
  with _lock:
1460
+ _write_run_record(run)
1461
+ _invalidate_runs_cache()
 
1462
 
1463
 
1464
  def update_run_if_owned(
 
1468
  mutator: Callable[[dict], None],
1469
  ) -> dict | None:
1470
  """Atomically mutate a running job only for its current lease owner."""
 
1471
  with _lock:
1472
  path = RUNS_DIR / f"{run_id}.json"
1473
  run = _read_json(path, None)
 
1478
  or run.get("lease_token") != lease_token):
1479
  return None
1480
  mutator(run)
1481
+ # The mutator may mark the run completed/failed; archiving happens in
1482
+ # the shared writer so this path stays in sync with save_run.
1483
+ _write_run_record(run)
1484
+ _invalidate_runs_cache()
1485
  return run
1486
 
1487
 
 
1502
  ``workflow_revision`` 时,queued/running 仍视为占用,completed 则
1503
  允许第一次迁移到带版本的幂等记录。
1504
  """
 
1505
  with _lock:
1506
  revision = run.get("workflow_revision")
1507
  for existing in list_runs():
 
1516
  existing["status"] = "superseded"
1517
  existing["finished_at"] = utcnow_iso()
1518
  existing["error_log"] = "Superseded by a newer upload or reprocess request"
1519
+ _write_run_record(existing)
1520
+ _invalidate_runs_cache()
 
1521
  continue
1522
  if same_revision or legacy_active:
1523
  if existing.get("status") in ("queued", "running"):
1524
  return existing, False
1525
  if existing.get("status") == "completed" and not allow_completed_rerun:
1526
  return existing, False
1527
+ _write_run_record(run)
1528
+ _invalidate_runs_cache()
 
1529
  return run, True
1530
 
1531
 
1532
  def delete_run(run_id: str) -> None:
1533
+ """Remove a run from the live queue and from the archive."""
1534
  _remove_json(RUNS_DIR / f"{run_id}.json")
1535
+ _remove_json(RUNS_ARCHIVE_DIR / f"{run_id}.json")
1536
+ _invalidate_runs_cache()
processor/app/processing/modules/lerobot_export.py CHANGED
@@ -21,9 +21,20 @@ class LeRobotExportModule(ProcessingModule):
21
  color = EXPORT_COLOR
22
  inputs = ({"key": "data", "label": "Exportable Data"},)
23
  outputs = ({"key": "dataset", "label": "Dataset"},)
24
- default_config = {"version": "v3.0", "split_ratio": 0.9, "shard_size": 100000}
 
 
 
 
 
 
 
 
 
25
  config_schema = (
26
  field("version", "select", "Version", "v3.0", options=["v2.1", "v3.0"]),
 
 
27
  )
28
  execution_target = "worker"
29
  capabilities = ("dataset_export", "lerobot")
@@ -80,6 +91,12 @@ class LeRobotExportModule(ProcessingModule):
80
  None)
81
  # 数据集先输出到 worker 临时目录,API 完成阶段会把导出产品放到
82
  # state/exports,不在项目根目录新增 processed/。
 
 
 
 
 
 
83
  dataset_dir = build_lerobot_dataset(episode_id, [episode_id],
84
  ctx.output_root, split_ratio,
85
  include_video_keys=include_video_keys,
@@ -87,7 +104,8 @@ class LeRobotExportModule(ProcessingModule):
87
  hand_3d_paths=hand_3d_paths,
88
  hand_3d_right_paths=hand_3d_right_paths,
89
  version=version,
90
- hand_3d_unit=hand_3d_unit)
 
91
  info = dataset_dir / "meta" / "info.json"
92
  if not info.exists():
93
  ctx.skip("Dataset build produced no meta/info.json")
 
21
  color = EXPORT_COLOR
22
  inputs = ({"key": "data", "label": "Exportable Data"},)
23
  outputs = ({"key": "dataset", "label": "Dataset"},)
24
+ # 默认排除 UMI 触觉力矩阵:每帧 250×250×3(187500 个整数),实测占
25
+ # canonical parquet 体积的九成以上,且 99.99% 是零 —— 训练用不到,
26
+ # 却让单集导出从 2.5 秒涨到 57 秒(写盘 + 逐元素统计 + 结果包上传),
27
+ # 结果 zip 也从几 MB 涨到几十 MB。要保留就把它从配置里删掉。
28
+ _DEFAULT_EXCLUDE = ",".join((
29
+ "observation.gripper_left_force_matrix",
30
+ "observation.gripper_right_force_matrix",
31
+ ))
32
+ default_config = {"version": "v3.0", "split_ratio": 0.9,
33
+ "shard_size": 100000, "exclude_columns": _DEFAULT_EXCLUDE}
34
  config_schema = (
35
  field("version", "select", "Version", "v3.0", options=["v2.1", "v3.0"]),
36
+ field("exclude_columns", "string", "Exclude columns (comma separated)",
37
+ _DEFAULT_EXCLUDE),
38
  )
39
  execution_target = "worker"
40
  capabilities = ("dataset_export", "lerobot")
 
91
  None)
92
  # 数据集先输出到 worker 临时目录,API 完成阶段会把导出产品放到
93
  # state/exports,不在项目根目录新增 processed/。
94
+ # 排除列:配置里逗号分隔。留空 = 全量导出(旧行为)。
95
+ exclude_columns = [
96
+ name.strip() for name
97
+ in str(ctx.config.get("exclude_columns", "") or "").split(",")
98
+ if name.strip()
99
+ ]
100
  dataset_dir = build_lerobot_dataset(episode_id, [episode_id],
101
  ctx.output_root, split_ratio,
102
  include_video_keys=include_video_keys,
 
104
  hand_3d_paths=hand_3d_paths,
105
  hand_3d_right_paths=hand_3d_right_paths,
106
  version=version,
107
+ hand_3d_unit=hand_3d_unit,
108
+ exclude_columns=exclude_columns)
109
  info = dataset_dir / "meta" / "info.json"
110
  if not info.exists():
111
  ctx.skip("Dataset build produced no meta/info.json")
processor/app/processing/modules/umi_slam_action.py ADDED
@@ -0,0 +1,192 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """UMI Slam Action — 从 SLAM 位姿派生 imitation learning 的 action 列。
2
+
3
+ UMI 采集端不录 action(写的是占位全零,导出时被恒定列过滤器剔除),
4
+ 导致数据集无法训练。本模块读取 UMI 夹爪的 SLAM 位姿与夹爪开合,
5
+ 按 UMI 官方口径反推出 7 维 action 并合并回 canonical:
6
+
7
+ Δp = R_tᵀ · (p_{t+1} − p_t) # 平移增量, 米
8
+ Δr = rotvec(R_tᵀ · R_{t+1}) # 旋转增量, 弧度
9
+ g = gripper_state[t+1, 0] / 100 # 夹爪开合, 0–1
10
+
11
+ 纯计算在 app/umi_slam_action.py(不依赖框架,便于单测与跨格式复用)。
12
+ """
13
+
14
+ from pathlib import Path
15
+
16
+ from app.processing import ProcessingModule, JobContext, ArtifactRef
17
+ from app.processing.registry import register
18
+ from app.processing.theme import HAND3D_COLOR
19
+
20
+ # 本模块产出的 artifact kind。合并白名单在 project_dataset 里单独维护,
21
+ # 新增 kind 必须同步加进那张表,否则列不会被并入 canonical。
22
+ ACTION_KIND = "umi_slam_action"
23
+
24
+ # 位姿只读 slam_trajectory:它是采集端的高频样本缓冲,丢帧时相邻格子里
25
+ # 仍有真实样本;而 slam_pose 在同样位置直接写零,只能插值硬补。
26
+ _TRAJECTORY_COLUMN = "observation.slam_trajectory"
27
+ _TIMESTAMP_COLUMN = "timestamp"
28
+ _GRIPPER_COLUMN = "observation.gripper_state"
29
+
30
+
31
+ @register
32
+ class UmiSlamActionModule(ProcessingModule):
33
+ slug = ACTION_KIND
34
+ version = "1.0"
35
+ category = "process"
36
+ label = "UMI Slam Action"
37
+ # 语义是"位姿 → 增量"的坐标变换,与同组做几何变换的手部模块一致;
38
+ # sync 也贴合"由轨迹反推动作"。必须取自 iconify-preload.js 的白名单
39
+ # (图标离线预载,不在表内的名字渲染为空白)。
40
+ icon = "ant-design:sync-outlined"
41
+ color = HAND3D_COLOR
42
+ # 端口 key 必须与上游 UMI Gripper 的输出 key 逐字一致 —— 前端对
43
+ # 类型化端口按 ``out.key === inp.key`` 校验,改名会导致连线被拒。
44
+ inputs = (
45
+ {"key": "slam_trajectory", "label": "SLAM Trajectory"},
46
+ {"key": "gripper_state", "label": "ESP Gripper State"},
47
+ )
48
+ outputs = ({"key": "action", "label": "Action"},)
49
+ default_config = {}
50
+ config_schema = ()
51
+ execution_target = "worker"
52
+ capabilities = ("umi_gripper", "action_derivation")
53
+
54
+ def run(self, ctx: JobContext) -> dict[str, ArtifactRef] | None:
55
+ parquet = self._find_source_parquet(ctx)
56
+ if parquet is None:
57
+ ctx.skip("No parquet with SLAM pose found in batch")
58
+
59
+ from app.umi_slam_action import derive_action
60
+ import time
61
+ import numpy as np
62
+ import pandas as pd
63
+
64
+ # 读取重试:采集端可能还在往同一项目目录写新集(每上传一集都会重写
65
+ # 项目数据树),worker 恰好读到写了一半的文件就会抛异常。实测
66
+ # ep52/ep71 都因此被误判为"列不可读"而跳过 —— 几秒后同一文件完全
67
+ # 正常。瞬时故障不该等同于永久失败,故重试若干次。
68
+ columns = [_TRAJECTORY_COLUMN, _TIMESTAMP_COLUMN, _GRIPPER_COLUMN]
69
+ frame = None
70
+ last_error: Exception | None = None
71
+ for attempt in range(5):
72
+ try:
73
+ frame = pd.read_parquet(parquet, columns=columns)
74
+ break
75
+ except Exception as exc:
76
+ last_error = exc
77
+ time.sleep(1.0 + attempt)
78
+ if frame is None:
79
+ # 带上路径与大小:同样的 ArrowInvalid 可能来自「文件被截断」或
80
+ # 「根本不是 parquet」,只看异常分不出来,必须知道读的是哪个文件。
81
+ try:
82
+ size = parquet.stat().st_size
83
+ head = parquet.open("rb").read(4)
84
+ detail = f"size={size}B head={head!r}"
85
+ except OSError as exc:
86
+ detail = f"stat failed: {exc}"
87
+ ctx.skip(f"SLAM columns unreadable after retries: "
88
+ f"{type(last_error).__name__}: {last_error} | "
89
+ f"file={parquet} ({detail})")
90
+
91
+ if _TRAJECTORY_COLUMN not in frame.columns:
92
+ # 非 UMI 批次(如 D435 手套、S80C)没有 SLAM 缓冲 —— 跳过,
93
+ # 不写入任何列,保证其他设备的导出行为完全不变。
94
+ ctx.skip("No SLAM trajectory column — not a UMI gripper batch")
95
+
96
+ total = len(frame)
97
+ stamps = (frame[_TIMESTAMP_COLUMN].to_numpy(dtype=float)
98
+ if _TIMESTAMP_COLUMN in frame.columns else np.arange(total) / 30.0)
99
+ derived = derive_action(
100
+ frame[_TRAJECTORY_COLUMN].tolist(),
101
+ stamps,
102
+ frame[_GRIPPER_COLUMN].tolist()
103
+ if _GRIPPER_COLUMN in frame.columns else [],
104
+ total,
105
+ )
106
+ if not derived:
107
+ ctx.skip("No usable SLAM samples (all frames dropped)")
108
+
109
+ # 输出按原 frame_index 对齐并保持升序:_merge_processing_frame 按
110
+ # frame_index 匹配,乱序会让列���位。
111
+ frames = sorted(derived)
112
+ ctx.progress(0.5)
113
+ output = ctx.output_dir / "umi_slam_action.parquet"
114
+ # 必须写 np.float32:pyarrow 对 Python float 推断 double,而 LeRobot
115
+ # 的 action 声明是 float32 —— 声明与实际 schema 不一致会让官方加载器
116
+ # 读出不匹配的类型。与导出端 _cast_float32 的做法保持一致。
117
+ pd.DataFrame({
118
+ "frame_index": [int(f) for f in frames],
119
+ "action": [[np.float32(value) for value in derived[f]["action"]]
120
+ for f in frames],
121
+ # ACT 的观测需要 proprioception:只有图像时模型不知道夹爪当前
122
+ # 在哪。这里的 observation.state 是相对本集起点的位置 + 夹爪开合。
123
+ "observation.state": [
124
+ [np.float32(value) for value in derived[f]["observation.state"]]
125
+ for f in frames],
126
+ }).to_parquet(output, index=False)
127
+ ctx.progress(1.0)
128
+
129
+ covered = len(frames)
130
+ return {
131
+ "action": ctx.ref(ACTION_KIND, output, metadata={
132
+ "column": "action",
133
+ "shape": [7],
134
+ "names": ["delta_x", "delta_y", "delta_z",
135
+ "rot_x", "rot_y", "rot_z", "gripper"],
136
+ "frames_covered": covered,
137
+ "frames_total": int(total),
138
+ "note": ("UMI 口径 action:当前夹爪局部系增量位姿 + 夹爪开合。"
139
+ "由 SLAM 位姿派生,零位姿丢帧已插值修复。"),
140
+ }),
141
+ }
142
+
143
+ @staticmethod
144
+ def _find_source_parquet(ctx: JobContext) -> Path | None:
145
+ """定位含 SLAM 位姿的 canonical parquet。
146
+
147
+ 优先用上游 ``slam_trajectory`` 端口连过来的产物(工作流连接驱动);
148
+ 未连或产物不可用时回退到 ``ctx.find_parquet()`` 扫描批次,
149
+ 保证"只连一根线也能跑"。
150
+
151
+ 上游 ref **必须验证是 parquet**:worker 组装 incoming 时,若边的
152
+ sourceHandle 对不上上游任何输出端口,会把该节点的**全部输出平铺**
153
+ 进来(runner `_incoming_artifacts` 规则 3),视频 ref 也会挂到
154
+ ``slam_trajectory`` 这个键上。实测因此把 mp4 当 parquet 读,报
155
+ ``magic bytes not found in footer`` —— 只看键名不验证就会中招。
156
+
157
+ 回退扫描**必须按 episode_id 过滤**:批次目录里往往躺着几十集的
158
+ canonical parquet,直接取第一个会静默用错集的数据(实测表现为
159
+ 每集算出的 action 完全相同)。worker 打包输入时通常只含一集,
160
+ 但本地重跑/整目录传入时就不是了。
161
+ """
162
+ import pyarrow.parquet as pq
163
+
164
+ def _is_slam_parquet(candidate: Path) -> bool:
165
+ if candidate.suffix.lower() != ".parquet" or not candidate.is_file():
166
+ return False
167
+ try:
168
+ return _TRAJECTORY_COLUMN in set(pq.read_schema(candidate).names)
169
+ except Exception:
170
+ return False
171
+
172
+ for key, ref in (ctx.incoming or {}).items():
173
+ if not str(key).startswith("slam_trajectory"):
174
+ continue
175
+ resolved = ctx.resolve(ref)
176
+ if resolved and _is_slam_parquet(resolved):
177
+ return resolved
178
+
179
+ episode_id = str((ctx.job or {}).get("episode_id") or "")
180
+ suffix = ""
181
+ if "_" in episode_id:
182
+ tail = episode_id.rsplit("_", 1)[-1]
183
+ if tail.isdigit():
184
+ suffix = f"episode_{int(tail):06d}.parquet"
185
+ candidates = ctx.find_parquet()
186
+ # 先只看文件名匹配本集的那一个
187
+ ordered = ([c for c in candidates if c.name == suffix] if suffix else [])
188
+ ordered += [c for c in candidates if c.name != suffix]
189
+ for candidate in ordered:
190
+ if _is_slam_parquet(candidate):
191
+ return candidate
192
+ return None
processor/app/project_dataset.py CHANGED
@@ -12,10 +12,13 @@ migration command and the upload path so both produce the same layout.
12
 
13
  from __future__ import annotations
14
 
 
15
  import json
16
  import os
17
  import re
18
  import shutil
 
 
19
  from datetime import datetime, timezone
20
  from pathlib import Path
21
  from typing import Any, Iterable
@@ -442,7 +445,42 @@ def episode_index_from_name(name: str) -> int | None:
442
  return int(match.group(1)) if match else None
443
 
444
 
445
- def project_episode_rows(project_root: Path) -> list[dict[str, Any]]:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
446
  rows: list[dict[str, Any]] = []
447
  for path in _episode_parquet_files(project_root):
448
  try:
@@ -455,6 +493,63 @@ def project_episode_rows(project_root: Path) -> list[dict[str, Any]]:
455
  return rows
456
 
457
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
458
  def _match_episode_row(rows: list[dict[str, Any]],
459
  episode_id: str) -> dict[str, Any] | None:
460
  """Match an episode row by canonical id first, then trailing index.
@@ -519,31 +614,86 @@ def _chunk_number_from_path(path: Path) -> int:
519
  return 0
520
 
521
 
522
- def episode_files(project_root: Path, episode_index: int) -> dict[str, Any]:
523
- """Return only one episode's source files from a project dataset."""
 
 
524
  root = Path(project_root)
525
- data_files = [
526
- path for path in (root / "data").rglob("*.parquet")
527
- if path.is_file() and _episode_number_from_path(path) == episode_index
528
- ] if (root / "data").is_dir() else []
529
- videos: list[tuple[str, Path]] = []
 
 
 
 
 
530
  for source, path in iter_video_streams(root / "videos"):
531
- if _episode_number_from_path(path) == episode_index:
532
- videos.append((source, path))
533
- row = next((item for item in project_episode_rows(root)
534
- if _episode_index(item.get("episode_index")) == episode_index), {})
535
- episode_id = _episode_id_for_row(row, root.name)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
536
  relative_meta: list[Path] = [Path("meta/info.json"), Path("meta/tasks.json")]
537
  return {
538
  "episode_index": episode_index,
539
- "episode_id": episode_id,
540
  "row": row,
541
- "data": data_files,
542
- "videos": videos,
543
  "meta": [path for path in relative_meta if (root / path).is_file()],
544
  }
545
 
546
 
 
 
 
 
 
 
 
 
 
 
547
  def _processing_token(value: Any, fallback: str = "result") -> str:
548
  text = re.sub(r"[^A-Za-z0-9_.-]+", "_", str(value or "")).strip("._-")
549
  return text or fallback
@@ -706,7 +856,8 @@ def publish_processing_result(
706
  records = _processing_artifacts(outputs)
707
  tabular_records = [item for item in records
708
  if str(item[2].get("kind") or "") in
709
- {"hand_keypoints", "hand_3d", "glove_sensor"}]
 
710
  # A reprocess is authoritative for its hand-3D sources. Remove any
711
  # source-qualified hand-3D columns left by an earlier partial publish
712
  # (including legacy ``*_1`` collision columns) before merging the fresh
@@ -717,10 +868,26 @@ def publish_processing_result(
717
  if str(name).startswith("processing.hand_3d.")]
718
  if stale_hand3d:
719
  data_frame = data_frame.drop(columns=stale_hand3d, errors="ignore")
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
720
  assigned_columns: set[str] = set(data_frame.columns)
721
  merged_columns: list[str] = []
722
  published = {"artifacts": {}}
723
  merged_hand3d_sources: set[str] = set()
 
724
 
725
  for node_id, handle, ref in records:
726
  source_path = _processing_ref_path(Path(extracted_root), ref)
@@ -734,6 +901,24 @@ def publish_processing_result(
734
  published_ref = dict(ref)
735
  source = _processing_token(ref.get("source_key") or metadata.get("source_key")
736
  or handle, kind)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
737
  # Review/export nodes pass the node-36 hand-3D refs through again.
738
  # They are the same source data, not a second measurement. Merge each
739
  # source once and keep the passthrough artifact only as an audit link.
@@ -760,7 +945,7 @@ def publish_processing_result(
760
  "storage": "browser_overlay"}
761
  published["artifacts"].setdefault(node_id, {})[handle] = published_ref
762
  continue
763
- if kind in {"hand_keypoints", "hand_3d", "glove_sensor"}:
764
  result_frame = pd.read_parquet(source_path)
765
  raw_columns = [str(name) for name in result_frame.columns
766
  if name not in {"frame_index", "episode_index"}]
 
12
 
13
  from __future__ import annotations
14
 
15
+ import copy
16
  import json
17
  import os
18
  import re
19
  import shutil
20
+ import threading
21
+ import time
22
  from datetime import datetime, timezone
23
  from pathlib import Path
24
  from typing import Any, Iterable
 
445
  return int(match.group(1)) if match else None
446
 
447
 
448
+ # ── Project index cache ───────────────────────────────
449
+ # Every _canonical_episode_data_files() call reads the complete episode
450
+ # metadata parquet set (52 episodes = 52 remote reads) plus two project-wide
451
+ # directory walks — measured at 1.4s warm, and the review page issues ~11
452
+ # video requests per episode open, so the same index was rebuilt a dozen
453
+ # times per click. Cache the parsed rows and the file index against a cheap
454
+ # fingerprint of the episode index files.
455
+ _PROJECT_CACHE_LOCK = threading.RLock()
456
+ _PROJECT_CACHE: dict[str, dict[str, Any]] = {}
457
+ # Computing the fingerprint still stats every episode index file, so a short
458
+ # TTL skips even that inside one page load. Structural changes are announced
459
+ # explicitly through invalidate_project_cache().
460
+ _PROJECT_CACHE_TTL = 60.0
461
+ _PROJECT_SCAN_LOCK = threading.Lock()
462
+
463
+
464
+ def _project_source_revision(project_root: Path) -> tuple:
465
+ """Fingerprint the episode index files (name, size, mtime).
466
+
467
+ The index files are written by whoever writes the data (upload, workflow
468
+ merge), so a change there means the project changed. ``data/`` and
469
+ ``videos/`` are deliberately not walked: any addition or removal there is
470
+ accompanied by an index write.
471
+ """
472
+ parts: list[tuple] = []
473
+ try:
474
+ for path in _episode_parquet_files(project_root):
475
+ stat = path.stat()
476
+ parts.append((path.name, stat.st_size, stat.st_mtime_ns))
477
+ except OSError:
478
+ return ()
479
+ return tuple(parts)
480
+
481
+
482
+ def _read_project_rows(project_root: Path) -> list[dict[str, Any]]:
483
+ """Uncached: one parquet read per episode, sorted by episode index."""
484
  rows: list[dict[str, Any]] = []
485
  for path in _episode_parquet_files(project_root):
486
  try:
 
493
  return rows
494
 
495
 
496
+ def _project_cache_entry(project_root: Path) -> dict[str, Any]:
497
+ """Return the cached ``{rows, index}`` for a project, rebuilding if stale."""
498
+ root = Path(project_root)
499
+ key = str(root)
500
+ now = time.monotonic()
501
+ with _PROJECT_CACHE_LOCK:
502
+ entry = _PROJECT_CACHE.get(key)
503
+ if entry is not None and now - entry["checked_at"] < _PROJECT_CACHE_TTL:
504
+ return entry
505
+ # The fingerprint touches the remote mount, so keep it outside the lock.
506
+ revision = _project_source_revision(root)
507
+ with _PROJECT_CACHE_LOCK:
508
+ entry = _PROJECT_CACHE.get(key)
509
+ if entry is not None and entry["revision"] == revision:
510
+ entry["checked_at"] = now
511
+ return entry
512
+ # Single-flight: a page load fires its requests in parallel, and they all
513
+ # miss at once. One rebuild serves the whole burst.
514
+ with _PROJECT_SCAN_LOCK:
515
+ now = time.monotonic()
516
+ with _PROJECT_CACHE_LOCK:
517
+ entry = _PROJECT_CACHE.get(key)
518
+ if entry is not None and (
519
+ entry["revision"] == revision
520
+ or now - entry["checked_at"] < _PROJECT_CACHE_TTL):
521
+ return entry
522
+ rows = _read_project_rows(root)
523
+ index = _build_project_file_index(root, rows)
524
+ entry = {
525
+ "revision": revision,
526
+ "checked_at": time.monotonic(),
527
+ "rows": rows,
528
+ "index": index,
529
+ }
530
+ with _PROJECT_CACHE_LOCK:
531
+ _PROJECT_CACHE[key] = entry
532
+ return entry
533
+
534
+
535
+ def invalidate_project_cache(project_root: Path | None = None) -> None:
536
+ """Drop cached project indexes after a structural change.
537
+
538
+ Called from ``localstore.invalidate_session_cache`` so an upload, delete or
539
+ rename becomes visible on the next read instead of after the TTL.
540
+ """
541
+ with _PROJECT_CACHE_LOCK:
542
+ if project_root is None:
543
+ _PROJECT_CACHE.clear()
544
+ else:
545
+ _PROJECT_CACHE.pop(str(Path(project_root)), None)
546
+
547
+
548
+ def project_episode_rows(project_root: Path) -> list[dict[str, Any]]:
549
+ # Callers may assume they own the rows, so never hand out the cache itself.
550
+ return copy.deepcopy(_project_cache_entry(project_root)["rows"])
551
+
552
+
553
  def _match_episode_row(rows: list[dict[str, Any]],
554
  episode_id: str) -> dict[str, Any] | None:
555
  """Match an episode row by canonical id first, then trailing index.
 
614
  return 0
615
 
616
 
617
+ def _build_project_file_index(
618
+ project_root: Path, rows: list[dict[str, Any]] | None = None
619
+ ) -> dict[int, dict[str, Any]]:
620
+ """Uncached: one pass over ``data/``, ``videos/`` and the episode rows."""
621
  root = Path(project_root)
622
+ index: dict[int, dict[str, Any]] = {}
623
+ data_root = root / "data"
624
+ if data_root.is_dir():
625
+ for path in data_root.rglob("*.parquet"):
626
+ if not path.is_file():
627
+ continue
628
+ number = _episode_number_from_path(path)
629
+ if number is None:
630
+ continue
631
+ index.setdefault(number, {"data": []})["data"].append(path)
632
  for source, path in iter_video_streams(root / "videos"):
633
+ number = _episode_number_from_path(path)
634
+ if number is None:
635
+ continue
636
+ index.setdefault(number, {"data": []}).setdefault(
637
+ "videos", []).append((source, path))
638
+ if rows is None:
639
+ rows = _read_project_rows(root)
640
+ for row in rows:
641
+ number = _episode_index(row.get("episode_index"))
642
+ if number is None:
643
+ continue
644
+ # ``episode_files`` resolves a duplicate episode number to the first
645
+ # row in index order; ``setdefault`` keeps that same winner.
646
+ index.setdefault(number, {"data": []}).setdefault("row", row)
647
+ return index
648
+
649
+
650
+ def project_episode_file_index(project_root: Path) -> dict[int, dict[str, Any]]:
651
+ """Index every episode's source files with a single pass over the project.
652
+
653
+ :func:`episode_files` re-walks ``data/``, ``videos/`` and the whole episode
654
+ index for each episode it is asked about, so scanning a project costs
655
+ ``(1 + episodes) x index files`` parquet reads on the (possibly remote)
656
+ storage mount. This entry point keeps that contract while serving repeat
657
+ callers from the revision-checked project cache.
658
+
659
+ Keys are episode numbers; values carry the same ``data`` / ``videos`` /
660
+ ``row`` payload :func:`episode_files` reports, and are ordered exactly as
661
+ a per-episode walk would have produced them.
662
+ """
663
+ return copy.deepcopy(_project_cache_entry(Path(project_root))["index"])
664
+
665
+
666
+ def episode_files_from_index(project_root: Path, episode_index: int,
667
+ index: dict[int, dict[str, Any]]) -> dict[str, Any]:
668
+ """Look one episode up in a prebuilt :func:`project_episode_file_index`.
669
+
670
+ Returns the same shape as :func:`episode_files`; the caller owns the index
671
+ lifetime and is responsible for rebuilding it when the project changes.
672
+ """
673
+ root = Path(project_root)
674
+ entry = index.get(episode_index) or {}
675
+ row = entry.get("row") or {}
676
  relative_meta: list[Path] = [Path("meta/info.json"), Path("meta/tasks.json")]
677
  return {
678
  "episode_index": episode_index,
679
+ "episode_id": _episode_id_for_row(row, root.name),
680
  "row": row,
681
+ "data": list(entry.get("data") or []),
682
+ "videos": list(entry.get("videos") or []),
683
  "meta": [path for path in relative_meta if (root / path).is_file()],
684
  }
685
 
686
 
687
+ def episode_files(project_root: Path, episode_index: int) -> dict[str, Any]:
688
+ """Return only one episode's source files from a project dataset.
689
+
690
+ Single-episode callers keep this entry point; it builds a one-shot index,
691
+ so it now walks the project once instead of three times.
692
+ """
693
+ return episode_files_from_index(
694
+ project_root, episode_index, project_episode_file_index(project_root))
695
+
696
+
697
  def _processing_token(value: Any, fallback: str = "result") -> str:
698
  text = re.sub(r"[^A-Za-z0-9_.-]+", "_", str(value or "")).strip("._-")
699
  return text or fallback
 
856
  records = _processing_artifacts(outputs)
857
  tabular_records = [item for item in records
858
  if str(item[2].get("kind") or "") in
859
+ {"hand_keypoints", "hand_3d", "glove_sensor",
860
+ "umi_slam_action"}]
861
  # A reprocess is authoritative for its hand-3D sources. Remove any
862
  # source-qualified hand-3D columns left by an earlier partial publish
863
  # (including legacy ``*_1`` collision columns) before merging the fresh
 
868
  if str(name).startswith("processing.hand_3d.")]
869
  if stale_hand3d:
870
  data_frame = data_frame.drop(columns=stale_hand3d, errors="ignore")
871
+ # Derived action replaces the collector's placeholder column instead of
872
+ # being renamed to ``processing.<kind>.*.action``. The UMI collector
873
+ # writes an all-zero ``action`` that the LeRobot exporter correctly drops
874
+ # as a constant/zero-information column, so without this drop the fresh
875
+ # action would land under a name no exporter reads. Only drop it when a
876
+ # umi_slam_action result is actually being published for this episode.
877
+ if any(str(ref.get("kind") or "") == "umi_slam_action"
878
+ for _node, _handle, ref in records):
879
+ # 本模块产出的列全部按原名覆盖(重跑幂等):action 之外还有
880
+ # observation.state,它同样不该因为二次合并被改名成 processing.*。
881
+ stale_action = [str(name) for name in data_frame.columns
882
+ if str(name) in {"action", "observation.state"}
883
+ or str(name).startswith("processing.umi_slam_action.")]
884
+ if stale_action:
885
+ data_frame = data_frame.drop(columns=stale_action, errors="ignore")
886
  assigned_columns: set[str] = set(data_frame.columns)
887
  merged_columns: list[str] = []
888
  published = {"artifacts": {}}
889
  merged_hand3d_sources: set[str] = set()
890
+ merged_action_files: set[str] = set()
891
 
892
  for node_id, handle, ref in records:
893
  source_path = _processing_ref_path(Path(extracted_root), ref)
 
901
  published_ref = dict(ref)
902
  source = _processing_token(ref.get("source_key") or metadata.get("source_key")
903
  or handle, kind)
904
+ # A review/review-export node forwards its upstream action ref, so the
905
+ # same derived action arrives twice (once from the producing node,
906
+ # once forwarded). Merging both would add a duplicate
907
+ # ``processing.<kind>.*.action`` column holding identical values.
908
+ # Deduplicate on the artifact file, not the node id: two genuinely
909
+ # separate umi_slam_action nodes produce different files and must both
910
+ # merge.
911
+ if kind == "umi_slam_action":
912
+ action_file = str(source_path.resolve())
913
+ if action_file in merged_action_files:
914
+ published_ref["path"] = str(data_path.relative_to(root)).replace("\\", "/")
915
+ published_ref["metadata"] = {
916
+ **metadata, "merged_into": published_ref["path"],
917
+ "deduplicated": True, "merged_columns": [],
918
+ }
919
+ published["artifacts"].setdefault(node_id, {})[handle] = published_ref
920
+ continue
921
+ merged_action_files.add(action_file)
922
  # Review/export nodes pass the node-36 hand-3D refs through again.
923
  # They are the same source data, not a second measurement. Merge each
924
  # source once and keep the passthrough artifact only as an audit link.
 
945
  "storage": "browser_overlay"}
946
  published["artifacts"].setdefault(node_id, {})[handle] = published_ref
947
  continue
948
+ if kind in {"hand_keypoints", "hand_3d", "glove_sensor", "umi_slam_action"}:
949
  result_frame = pd.read_parquet(source_path)
950
  raw_columns = [str(name) for name in result_frame.columns
951
  if name not in {"frame_index", "episode_index"}]
processor/app/routes/devices.py CHANGED
@@ -1,7 +1,8 @@
1
  """Device heartbeat & polling API — 本地 JSON 存储(无数据库)。
2
 
3
  设备存 data/state/devices.json;采集端轮询 /device/tasks 获取
4
- active 项目列表(任务概念已移除,项目即采集任务)。
 
5
  """
6
 
7
  from __future__ import annotations
@@ -18,6 +19,8 @@ from app.localstore import (
18
  list_projects,
19
  scan_sessions,
20
  )
 
 
21
  from app.security import verify_api_key
22
  from app.device_naming import decorate_device_sources, display_names_for_sources
23
 
@@ -237,18 +240,31 @@ async def device_tasks(
237
  device_name: str = Query(..., min_length=1),
238
  _: str = Depends(verify_api_key),
239
  ):
240
- """采集端轮询此接口获取 active 项目(DAQ client 格式)。"""
 
 
 
 
 
 
241
  project_rows = await asyncio.to_thread(list_projects)
242
  projects = [p for p in project_rows if p.get("status", "active") == "active"]
243
- # Prefer the last complete snapshot. A cache miss only happens during a
244
- # cold start, and even that first remote scan is kept off the event loop.
245
- episodes = cached_sessions_for_tasks()
246
- if episodes is None:
247
- episodes = await asyncio.to_thread(scan_sessions)
 
 
 
 
 
248
 
249
  tasks = []
250
  for p in projects:
251
- cur = sum(1 for e in episodes if e.get("project") == p.get("name"))
 
 
252
  params = p.get("params") or {}
253
  if not isinstance(params, dict):
254
  params = {}
@@ -261,6 +277,37 @@ async def device_tasks(
261
  "current_count": cur,
262
  "assigned_at": p.get("created_at"),
263
  "params": params,
 
 
 
264
  })
265
 
266
  return {"tasks": tasks, "updated_at": utcnow().isoformat()}
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  """Device heartbeat & polling API — 本地 JSON 存储(无数据库)。
2
 
3
  设备存 data/state/devices.json;采集端轮询 /device/tasks 获取
4
+ active 项目列表(任务概念已移除,项目即采集任务);采集端「录制完成」的
5
+ 增量上报存 data/state/device_task_progress.json(见 app/device_progress.py)。
6
  """
7
 
8
  from __future__ import annotations
 
19
  list_projects,
20
  scan_sessions,
21
  )
22
+ from app import device_progress
23
+ from app.device_progress import reported_totals
24
  from app.security import verify_api_key
25
  from app.device_naming import decorate_device_sources, display_names_for_sources
26
 
 
240
  device_name: str = Query(..., min_length=1),
241
  _: str = Depends(verify_api_key),
242
  ):
243
+ """采集端轮询此接口获取 active 项目(DAQ client 格式)。
244
+
245
+ current_count 口径:各设备上报的录制完成数之和(见 app/device_progress.py);
246
+ 只有从未收到过上报的项目才回退到「项目目录下的 session 数」,让升级前就
247
+ 存在的历史项目不归零。session 是**上传**的产物,早先直接拿它当进度会让
248
+ 同一条数据被算两次(录制一次、上传一次)。
249
+ """
250
  project_rows = await asyncio.to_thread(list_projects)
251
  projects = [p for p in project_rows if p.get("status", "active") == "active"]
252
+ reported = await asyncio.to_thread(reported_totals)
253
+ # 只有需要 session 兜底时才扫目录(用了上报口径的项目不再触发远程遍历)
254
+ needs_sessions = device_progress.needs_session_fallback(projects, reported)
255
+ episodes: list = []
256
+ if needs_sessions:
257
+ # Prefer the last complete snapshot. A cache miss only happens during a
258
+ # cold start, and even that first remote scan is kept off the event loop.
259
+ episodes = cached_sessions_for_tasks()
260
+ if episodes is None:
261
+ episodes = await asyncio.to_thread(scan_sessions)
262
 
263
  tasks = []
264
  for p in projects:
265
+ sessions = (sum(1 for e in episodes if e.get("project") == p.get("name"))
266
+ if needs_sessions else None)
267
+ cur, source = device_progress.resolve_count(p, reported, sessions or 0)
268
  params = p.get("params") or {}
269
  if not isinstance(params, dict):
270
  params = {}
 
277
  "current_count": cur,
278
  "assigned_at": p.get("created_at"),
279
  "params": params,
280
+ # 供 Web UI/排查用:进度口径与该项目实际已上传的 session 数
281
+ "progress_source": source,
282
+ "session_count": sessions,
283
  })
284
 
285
  return {"tasks": tasks, "updated_at": utcnow().isoformat()}
286
+
287
+
288
+ @router.post("/device/tasks/progress")
289
+ async def device_task_progress(body: dict | None = None,
290
+ _: str = Depends(verify_api_key)):
291
+ """采集端「录制完成」增量上报(幂等)。
292
+
293
+ body::
294
+
295
+ {"task_id": "<项目 id>", "device_name": "EGO_001",
296
+ "session_id": "EGO_001:<task_id>:<本地水位>", "increment": 1}
297
+
298
+ 返回 ``{"completed_count": <该项目最新进度>}`` —— 采集端把这个数当后端
299
+ 权威数落进自己的分账(backend_count)。同一 (device_name, session_id)
300
+ 重放不重复计数:采集端崩溃/断网后重发同一个水位是安全的。
301
+
302
+ 错误:参数缺失/非法、任务不存在 → 400。**刻意不用 404** —— 采集端把 404
303
+ 当成「后端还没实现该端点」并永久降级(整机不再上报任何进度)。
304
+ """
305
+ from fastapi import HTTPException
306
+
307
+ projects = await asyncio.to_thread(list_projects)
308
+ known = {str(p.get("id") or "") for p in projects}
309
+ try:
310
+ return await asyncio.to_thread(
311
+ device_progress.apply_report, body or {}, known)
312
+ except ValueError as exc:
313
+ raise HTTPException(status_code=400, detail=str(exc))
processor/app/routes/export.py CHANGED
@@ -362,8 +362,15 @@ def _workflow_export_version(episode_id: str) -> str | None:
362
 
363
 
364
  @router.get("/episode-format/{episode_id}")
365
- async def episode_export_format(episode_id: str):
366
- """前端按钮徽标:该批次下载时实际返回的格式。"""
 
 
 
 
 
 
 
367
  if get_episode(episode_id) is None:
368
  raise HTTPException(status_code=404, detail="Episode not found")
369
  target = _episode_export_target(episode_id)
 
362
 
363
 
364
  @router.get("/episode-format/{episode_id}")
365
+ def episode_export_format(episode_id: str):
366
+ """前端按钮徽标:该批次下载时实际返回的格式。
367
+
368
+ 同步端点(FastAPI 丢线程池执行)。以前写成 ``async def`` 却调
369
+ :func:`_episode_export_target` —— 那里读历史 run 快照 + 逐个 ``is_file()``
370
+ 全是远程阻塞 I/O,在事件循环上跑会把同一次开页的其它请求一起冻住:
371
+ 实测该接口冷调用 4.4s,期间 hand-3d / slam / tactile 全部排队,这正是
372
+ "打开一条数据时而 2 秒时而 7 秒"的来源。
373
+ """
374
  if get_episode(episode_id) is None:
375
  raise HTTPException(status_code=404, detail="Episode not found")
376
  target = _episode_export_target(episode_id)
processor/app/routes/session.py CHANGED
@@ -489,6 +489,37 @@ async def _process_upload_job(
489
  is_depth_source, iter_video_streams, normalize_extracted_dataset,
490
  )
491
  normalize_extracted_dataset(staging_dir, batch_name)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
492
  info = _read_json(staging_dir / "meta" / "info.json")
493
  if not isinstance(info, dict):
494
  info = {}
@@ -573,6 +604,13 @@ async def _process_upload_job(
573
  )
574
  write_episode_state(batch_name, new_state)
575
  save_annotations(batch_name, [])
 
 
 
 
 
 
 
576
  # A new upload or a same-name re-upload changes files/cameras, not
577
  # just review state. Force the next session scan to discover the new
578
  # structure before a workflow or review request can use stale streams.
 
489
  is_depth_source, iter_video_streams, normalize_extracted_dataset,
490
  )
491
  normalize_extracted_dataset(staging_dir, batch_name)
492
+
493
+ # ── 内容级重复检测(与包名无关)──
494
+ # 采集端可能把已上传过的录制换个包名再推一次(本地去重标记丢失 /
495
+ # 包名规则变化),此时上面的 incoming_name 判断不出来,同一份数据
496
+ # 会被追加成新批次。用规范化后的内容指纹兜底:命中已有批次就按
497
+ # 同名重传处理,覆盖原批次而不是新增一条重复数据。
498
+ # 算不出指纹时返回 None,一律按新批次正常入库。
499
+ try:
500
+ from app.batch_fingerprint import (
501
+ compute_fingerprint, lookup_duplicate,
502
+ )
503
+ content_fingerprint = compute_fingerprint(staging_dir)
504
+ duplicate_of = lookup_duplicate(
505
+ project_folder, content_fingerprint, existing_ids)
506
+ except Exception as fingerprint_err:
507
+ content_fingerprint = None
508
+ duplicate_of = None
509
+ print(f"[Upload] Fingerprint skipped: {fingerprint_err}")
510
+ if duplicate_of and duplicate_of != batch_name:
511
+ print(
512
+ f"[Upload] Duplicate content of {duplicate_of} "
513
+ f"(incoming name {incoming_name!r}) → replacing that episode"
514
+ )
515
+ batch_name = duplicate_of
516
+ is_reupload = True
517
+ batch_preexisted = True
518
+ # staging 目录名只是本地上传中转名,append_project_episode 以
519
+ # batch_name 参数为准,无需跟着改名。
520
+ old_state = read_episode_state(batch_name)
521
+ old_annotations = list_annotations(batch_name)
522
+
523
  info = _read_json(staging_dir / "meta" / "info.json")
524
  if not isinstance(info, dict):
525
  info = {}
 
604
  )
605
  write_episode_state(batch_name, new_state)
606
  save_annotations(batch_name, [])
607
+ # 指纹只在批次真正提交后记录:失败的上传不该占住一个内容身份,
608
+ # 否则重试时会被判成"重复"而覆盖掉自己。
609
+ try:
610
+ from app.batch_fingerprint import record_fingerprint
611
+ record_fingerprint(project_folder, content_fingerprint, batch_name)
612
+ except Exception as fingerprint_err:
613
+ print(f"[Upload] Fingerprint record skipped: {fingerprint_err}")
614
  # A new upload or a same-name re-upload changes files/cameras, not
615
  # just review state. Force the next session scan to discover the new
616
  # structure before a workflow or review request can use stale streams.
processor/app/routes/video.py CHANGED
@@ -57,6 +57,21 @@ _TACTILE_FORCE_COLUMNS = {
57
  "left": "observation.gripper_left_force",
58
  "right": "observation.gripper_right_force",
59
  }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
60
  # Sequential decode is cheap, but decoding through more than this many
61
  # frames to reach a scrub target is not; jump via a seek reader instead.
62
  _DEPTH_WINDOW_SEEK_GAP = 6
@@ -276,7 +291,17 @@ def _read_gripper_state(value) -> list[float] | None:
276
 
277
 
278
  def _read_slam_rows(path: Path) -> tuple[str, list[dict]]:
279
- """Read canonical ``observation.slam_pose`` rows once per parquet revision."""
 
 
 
 
 
 
 
 
 
 
280
  import numpy as np
281
  import pandas as pd
282
 
@@ -290,23 +315,45 @@ def _read_slam_rows(path: Path) -> tuple[str, list[dict]]:
290
  cached = _SLAM_DATA_CACHE.get(cache_key)
291
  if cached is not None and cached[0] == revision:
292
  return cached
293
- columns = ["frame_index", "timestamp", "observation.slam_pose"]
294
- has_gripper_state = False
295
  try:
296
  import pyarrow.parquet as pq
297
  available_columns = set(pq.ParquetFile(path).schema_arrow.names)
298
- has_gripper_state = "observation.gripper_state" in available_columns
299
  except Exception:
300
- pass
301
- if has_gripper_state:
 
 
 
 
 
 
 
 
 
 
 
302
  columns.append("observation.gripper_state")
303
  try:
304
  df = pd.read_parquet(path, columns=columns, engine="pyarrow")
305
  except Exception:
306
  return revision, []
307
 
 
 
 
 
 
 
 
 
 
 
 
 
 
308
  rows: list[dict] = []
309
- for _, row in df.iterrows():
310
  try:
311
  frame = int(row.get("frame_index", len(rows)))
312
  except (TypeError, ValueError):
@@ -316,11 +363,16 @@ def _read_slam_rows(path: Path) -> tuple[str, list[dict]]:
316
  timestamp = float(timestamp) if np.isfinite(float(timestamp)) else None
317
  except (TypeError, ValueError):
318
  timestamp = None
319
- pose = row.get("observation.slam_pose")
320
- try:
321
- values = np.asarray(pose, dtype=np.float64).reshape(-1)
322
- except (TypeError, ValueError):
323
- values = np.empty(0, dtype=np.float64)
 
 
 
 
 
324
 
325
  # A missing native SLAM sample is represented by an all-zero pose in
326
  # some recordings. It is not the same thing as a valid pose at the
@@ -337,7 +389,7 @@ def _read_slam_rows(path: Path) -> tuple[str, list[dict]]:
337
  "p": [float(value) for value in values[:7]],
338
  "t": timestamp,
339
  "g": _read_gripper_state(row.get("observation.gripper_state"))
340
- if has_gripper_state else None,
341
  })
342
  rows.sort(key=lambda item: item["f"])
343
 
@@ -424,6 +476,53 @@ def _read_tactile_force_series(path: Path) -> dict | None:
424
  return None
425
 
426
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
427
  def _decode_tactile_matrix(value):
428
  """Decode one canonical 250x250x3 row-diff force matrix to mN.
429
 
@@ -432,8 +531,9 @@ def _decode_tactile_matrix(value):
432
  prefix sum must run over the full 750-value row before reshaping back to
433
  HWC. Splitting H/W/C first and accumulating each channel independently
434
  produces a different matrix and was the cause of the incorrect preview.
435
- The int16 cast is intentional: the recorder's contract is modulo-2^16 and
436
- the final values are integer mN.
 
437
  """
438
  import numpy as np
439
 
@@ -466,6 +566,73 @@ def _decode_tactile_matrix(value):
466
  return decoded
467
 
468
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
469
  def _read_tactile_frame(path: Path, frame_index: int, side: str):
470
  """Read only the requested parquet row and side.
471
 
@@ -504,7 +671,7 @@ def _read_tactile_frame(path: Path, frame_index: int, side: str):
504
  # making the UI show a broken image.
505
  if decoded is None:
506
  return np.zeros((3, *_TACTILE_SHAPE), dtype=np.float32)
507
- return decoded
508
  except Exception:
509
  return None
510
  return None
@@ -516,8 +683,8 @@ def _tactile_force_field(
516
  """Return one smoothed fz pressure field as float32 bytes.
517
 
518
  The payload is numeric data, not a rendered image. The frontend applies
519
- the color map in WebGL2. ``lo``/``hi`` preserve the acquisition-side
520
- per-frame contrast normalization without sending a rendered image.
521
  """
522
  import cv2
523
  import numpy as np
@@ -545,12 +712,7 @@ def _tactile_force_field(
545
  # pressure visualization.
546
  pressure = np.maximum(matrix[2], 0.0)
547
  pressure = cv2.GaussianBlur(pressure, (3, 3), 0.8)
548
- valid = pressure[pressure > 0.01]
549
- lo = 0.0
550
- hi = 1.0
551
- if valid.size > 50:
552
- lo = float(np.percentile(valid, 10))
553
- hi = max(float(np.percentile(valid, 98)) * 1.3, lo + 0.5)
554
  field = np.ascontiguousarray(pressure, dtype="<f4")
555
  result = (field.tobytes(), lo, hi)
556
  with _TACTILE_PREVIEW_LOCK:
@@ -567,11 +729,12 @@ _TACTILE_RANGE_CACHE_MAX = 24
567
 
568
  def _tactile_range_payload(path: Path, first: int, last: int,
569
  side: str) -> bytes | None:
570
- """Encode a run of consecutive fz fields as R8 plus per-frame min/max.
571
 
572
  Body layout: ``frames * 250 * 250`` bytes of R8 (one byte per cell, row
573
  major) followed by ``frames * 2`` little-endian float32 values holding
574
- each frame's ``lo``/``hi`` normalisation pair. One request per window
 
575
  replaces one 250 KB request per frame, which is what saturated the
576
  browser's socket buffers (net::ERR_NO_BUFFER_SPACE) during playback.
577
  """
@@ -616,20 +779,19 @@ def _tactile_range_payload(path: Path, first: int, last: int,
616
  pixels = _TACTILE_SHAPE[0] * _TACTILE_SHAPE[1]
617
  blocks: list[bytes] = []
618
  pairs: list[float] = []
 
 
 
 
619
  for frame in range(int(first), int(last)):
620
- matrix = _decode_tactile_matrix(rows.get(frame))
 
621
  if matrix is None:
622
  blocks.append(bytes(pixels))
623
- pairs.extend((0.0, 1.0))
624
  continue
625
  pressure = np.maximum(matrix[2], 0.0)
626
  pressure = cv2.GaussianBlur(pressure, (3, 3), 0.8)
627
- valid = pressure[pressure > 0.01]
628
- lo = 0.0
629
- hi = 1.0
630
- if valid.size > 50:
631
- lo = float(np.percentile(valid, 10))
632
- hi = max(float(np.percentile(valid, 98)) * 1.3, lo + 0.5)
633
  scaled = np.clip((pressure - lo) / (hi - lo + 1e-6), 0.0, 1.0)
634
  blocks.append(np.rint(scaled * 255.0).astype(np.uint8).tobytes())
635
  pairs.extend((lo, hi))
 
57
  "left": "observation.gripper_left_force",
58
  "right": "observation.gripper_right_force",
59
  }
60
+ # Colour-scale anchors for the force heatmaps. The range is measured over the
61
+ # whole episode, never per frame: a frame with no contact holds nothing above
62
+ # the noise threshold but noise, so a per-frame range collapses onto that noise
63
+ # and stretches every blip to the top of the colour ramp - the tile lights up
64
+ # with red dots that read as real pressure. ``lo`` is the episode's median
65
+ # valid pressure (its noise floor; anything at or below renders as background)
66
+ # and ``hi`` its peak plus headroom.
67
+ _TACTILE_NOISE_FLOOR_MN = 0.01
68
+ _TACTILE_RANGE_BASELINE_PERCENTILE = 50.0
69
+ _TACTILE_RANGE_PEAK_PERCENTILE = 99.9
70
+ _TACTILE_RANGE_HEADROOM = 1.3
71
+ _TACTILE_RANGE_MIN_SPAN_MN = 0.5
72
+ _TACTILE_RANGE_SAMPLE_FRAMES = 256
73
+ _TACTILE_SCALE_CACHE: OrderedDict[tuple, tuple[float, float]] = OrderedDict()
74
+ _TACTILE_SCALE_CACHE_MAX = 8
75
  # Sequential decode is cheap, but decoding through more than this many
76
  # frames to reach a scrub target is not; jump via a seek reader instead.
77
  _DEPTH_WINDOW_SEEK_GAP = 6
 
291
 
292
 
293
  def _read_slam_rows(path: Path) -> tuple[str, list[dict]]:
294
+ """Read canonical SLAM rows once per parquet revision.
295
+
296
+ 位姿有两个来源,取决于采集端版本:
297
+
298
+ - 旧版写 ``observation.slam_pose``(每帧一个 7 维位姿);
299
+ - 新版只写 ``observation.slam_trajectory``(高频样本缓冲),没有
300
+ ``slam_pose``。此时前端曾经整块空白 —— 因为这里硬编码读后者。
301
+
302
+ 两条路径统一成每帧一个位姿:trajectory 交给 ``poses_from_trajectory``
303
+ 重建(它已按时间戳取最近样本,丢帧时也能从相邻格子找回)。
304
+ """
305
  import numpy as np
306
  import pandas as pd
307
 
 
315
  cached = _SLAM_DATA_CACHE.get(cache_key)
316
  if cached is not None and cached[0] == revision:
317
  return cached
318
+
 
319
  try:
320
  import pyarrow.parquet as pq
321
  available_columns = set(pq.ParquetFile(path).schema_arrow.names)
 
322
  except Exception:
323
+ return revision, []
324
+
325
+ has_pose = "observation.slam_pose" in available_columns
326
+ has_trajectory = "observation.slam_trajectory" in available_columns
327
+ if not has_pose and not has_trajectory:
328
+ return revision, []
329
+
330
+ columns = ["frame_index", "timestamp"]
331
+ if has_pose:
332
+ columns.append("observation.slam_pose")
333
+ if has_trajectory:
334
+ columns.append("observation.slam_trajectory")
335
+ if "observation.gripper_state" in available_columns:
336
  columns.append("observation.gripper_state")
337
  try:
338
  df = pd.read_parquet(path, columns=columns, engine="pyarrow")
339
  except Exception:
340
  return revision, []
341
 
342
+ # 新版只有 trajectory:重建出逐帧位姿,让下游按同一套逻辑消费。
343
+ rebuilt = None
344
+ if not has_pose:
345
+ from app.umi_slam_action import poses_from_trajectory
346
+ stamps = (df["timestamp"].to_numpy(dtype=float)
347
+ if "timestamp" in df.columns else None)
348
+ if stamps is None:
349
+ stamps = np.arange(len(df), dtype=float) / 30.0
350
+ rebuilt = poses_from_trajectory(
351
+ df["observation.slam_trajectory"].tolist(), stamps, len(df))
352
+ if rebuilt is None:
353
+ return revision, []
354
+
355
  rows: list[dict] = []
356
+ for position, (_, row) in enumerate(df.iterrows()):
357
  try:
358
  frame = int(row.get("frame_index", len(rows)))
359
  except (TypeError, ValueError):
 
363
  timestamp = float(timestamp) if np.isfinite(float(timestamp)) else None
364
  except (TypeError, ValueError):
365
  timestamp = None
366
+ if has_pose:
367
+ pose = row.get("observation.slam_pose")
368
+ try:
369
+ values = np.asarray(pose, dtype=np.float64).reshape(-1)
370
+ except (TypeError, ValueError):
371
+ values = np.empty(0, dtype=np.float64)
372
+ else:
373
+ # 已从 trajectory 重建,按位置对齐取本帧位姿
374
+ values = (rebuilt[position] if position < len(rebuilt)
375
+ else np.empty(0, dtype=np.float64))
376
 
377
  # A missing native SLAM sample is represented by an all-zero pose in
378
  # some recordings. It is not the same thing as a valid pose at the
 
389
  "p": [float(value) for value in values[:7]],
390
  "t": timestamp,
391
  "g": _read_gripper_state(row.get("observation.gripper_state"))
392
+ if "observation.gripper_state" in available_columns else None,
393
  })
394
  rows.sort(key=lambda item: item["f"])
395
 
 
476
  return None
477
 
478
 
479
+ def _tactile_scale_factor(path: Path) -> float:
480
+ """Read ``features.*.scale`` for the force matrix from dataset metadata.
481
+
482
+ Newer recorders store the matrix as quantised integers scaled by 100
483
+ (``"scale": 100`` alongside ``units: mN``), so the raw values must be
484
+ divided by that factor to recover mN. Older recordings omit the field and
485
+ already store plain mN, so the default is 1.
486
+
487
+ The factor is read from metadata rather than hard-coded because the
488
+ recordings genuinely differ; assuming 100 would divide old data by 100.
489
+ """
490
+ try:
491
+ import json
492
+ # parquet 位于 <root>/data/chunk-000/,info.json 位于 <root>/meta/。
493
+ # 逐级上溯找 meta/info.json,避免依赖固定的目录深度。
494
+ root = Path(path).parent
495
+ for candidate in (root, *root.parents):
496
+ meta = candidate / "meta" / "info.json"
497
+ if meta.is_file():
498
+ info = json.loads(meta.read_text(encoding="utf-8"))
499
+ break
500
+ else:
501
+ return 1.0
502
+ except (OSError, ValueError, TypeError):
503
+ return 1.0
504
+ features = info.get("features") or {}
505
+ for key in _TACTILE_COLUMNS.values():
506
+ entry = features.get(key)
507
+ if not isinstance(entry, dict):
508
+ continue
509
+ try:
510
+ factor = float(entry.get("scale"))
511
+ except (TypeError, ValueError):
512
+ continue
513
+ # 0/负数会让整块力值失真,视为未声明。
514
+ if factor > 0:
515
+ return factor
516
+ return 1.0
517
+
518
+
519
+ def _apply_tactile_scale(matrix, factor: float):
520
+ """把量化整数还原成物理单位(mN);factor<=1 时不做任何改动。"""
521
+ if matrix is None or not factor or factor <= 1:
522
+ return matrix
523
+ return matrix / factor
524
+
525
+
526
  def _decode_tactile_matrix(value):
527
  """Decode one canonical 250x250x3 row-diff force matrix to mN.
528
 
 
531
  prefix sum must run over the full 750-value row before reshaping back to
532
  HWC. Splitting H/W/C first and accumulating each channel independently
533
  produces a different matrix and was the cause of the incorrect preview.
534
+ The int16 cast is intentional: the recorder's contract is modulo-2^16.
535
+ Values are quantised integers in ``units: mN``; the physical scale is
536
+ applied by callers via ``_tactile_scale_factor``.
537
  """
538
  import numpy as np
539
 
 
566
  return decoded
567
 
568
 
569
+ def _compute_tactile_display_range(path: Path, column: str) -> tuple[float, float]:
570
+ """Sample a whole episode and return its ``(lo, hi)`` colour range in mN.
571
+
572
+ Every stored frame shares one row group, so reading the column costs the
573
+ same as reading any slice of it; the sampling stride bounds the decode and
574
+ the pooled sample, not the I/O.
575
+ """
576
+ import cv2
577
+ import numpy as np
578
+ import pyarrow as pa
579
+ import pyarrow.parquet as pq
580
+
581
+ try:
582
+ table = pq.read_table(path, columns=[column])
583
+ except Exception:
584
+ return 0.0, 1.0
585
+ column_data = table.column(column)
586
+ if isinstance(column_data, pa.ChunkedArray):
587
+ column_data = column_data.combine_chunks()
588
+ offsets = np.asarray(column_data.offsets, dtype=np.int64)
589
+ values = np.asarray(column_data.values, dtype=np.int16)
590
+ frames = len(offsets) - 1
591
+ if frames <= 0:
592
+ return 0.0, 1.0
593
+ stride = max(1, frames // _TACTILE_RANGE_SAMPLE_FRAMES)
594
+ scale_factor = _tactile_scale_factor(path)
595
+ samples: list[np.ndarray] = []
596
+ for index in range(0, frames, stride):
597
+ matrix = _apply_tactile_scale(
598
+ _decode_tactile_matrix(values[offsets[index]:offsets[index + 1]]),
599
+ scale_factor)
600
+ if matrix is None:
601
+ continue
602
+ pressure = cv2.GaussianBlur(np.maximum(matrix[2], 0.0), (3, 3), 0.8)
603
+ valid = pressure[pressure > _TACTILE_NOISE_FLOOR_MN]
604
+ if valid.size:
605
+ samples.append(valid)
606
+ if not samples:
607
+ return 0.0, 1.0
608
+ pooled = np.concatenate(samples)
609
+ lo = float(np.percentile(pooled, _TACTILE_RANGE_BASELINE_PERCENTILE))
610
+ hi = max(float(np.percentile(pooled, _TACTILE_RANGE_PEAK_PERCENTILE))
611
+ * _TACTILE_RANGE_HEADROOM, lo + _TACTILE_RANGE_MIN_SPAN_MN)
612
+ return lo, hi
613
+
614
+
615
+ def _tactile_display_range(path: Path, revision: str,
616
+ side: str) -> tuple[float, float]:
617
+ """Return the cached episode-wide colour range ``(lo, hi)`` for one side."""
618
+ column = _TACTILE_COLUMNS.get(side)
619
+ if column is None:
620
+ return 0.0, 1.0
621
+ key = (str(path.resolve()), revision, side)
622
+ with _TACTILE_PREVIEW_LOCK:
623
+ cached = _TACTILE_SCALE_CACHE.get(key)
624
+ if cached is not None:
625
+ _TACTILE_SCALE_CACHE.move_to_end(key)
626
+ return cached
627
+ result = _compute_tactile_display_range(path, column)
628
+ with _TACTILE_PREVIEW_LOCK:
629
+ _TACTILE_SCALE_CACHE[key] = result
630
+ _TACTILE_SCALE_CACHE.move_to_end(key)
631
+ while len(_TACTILE_SCALE_CACHE) > _TACTILE_SCALE_CACHE_MAX:
632
+ _TACTILE_SCALE_CACHE.popitem(last=False)
633
+ return result
634
+
635
+
636
  def _read_tactile_frame(path: Path, frame_index: int, side: str):
637
  """Read only the requested parquet row and side.
638
 
 
671
  # making the UI show a broken image.
672
  if decoded is None:
673
  return np.zeros((3, *_TACTILE_SHAPE), dtype=np.float32)
674
+ return _apply_tactile_scale(decoded, _tactile_scale_factor(path))
675
  except Exception:
676
  return None
677
  return None
 
683
  """Return one smoothed fz pressure field as float32 bytes.
684
 
685
  The payload is numeric data, not a rendered image. The frontend applies
686
+ the color map in WebGL2. ``lo``/``hi`` are the episode-wide contrast
687
+ range, so the same mN value maps to the same colour in every frame.
688
  """
689
  import cv2
690
  import numpy as np
 
712
  # pressure visualization.
713
  pressure = np.maximum(matrix[2], 0.0)
714
  pressure = cv2.GaussianBlur(pressure, (3, 3), 0.8)
715
+ lo, hi = _tactile_display_range(path, f"{revision[0]:x}-{revision[1]:x}", side)
 
 
 
 
 
716
  field = np.ascontiguousarray(pressure, dtype="<f4")
717
  result = (field.tobytes(), lo, hi)
718
  with _TACTILE_PREVIEW_LOCK:
 
729
 
730
  def _tactile_range_payload(path: Path, first: int, last: int,
731
  side: str) -> bytes | None:
732
+ """Encode a run of consecutive fz fields as R8 plus the episode's min/max.
733
 
734
  Body layout: ``frames * 250 * 250`` bytes of R8 (one byte per cell, row
735
  major) followed by ``frames * 2`` little-endian float32 values holding
736
+ each frame's ``lo``/``hi`` normalisation pair - the episode-wide range,
737
+ repeated once per frame so the header count stays fixed. One request per window
738
  replaces one 250 KB request per frame, which is what saturated the
739
  browser's socket buffers (net::ERR_NO_BUFFER_SPACE) during playback.
740
  """
 
779
  pixels = _TACTILE_SHAPE[0] * _TACTILE_SHAPE[1]
780
  blocks: list[bytes] = []
781
  pairs: list[float] = []
782
+ # 量化整数先还原成 mN;色阶取整集区间,不逐帧重算,否则没有接触的帧会
783
+ # 拿自身噪声当满量程,把噪点画成接触点。
784
+ scale_factor = _tactile_scale_factor(path)
785
+ lo, hi = _tactile_display_range(path, revision, side)
786
  for frame in range(int(first), int(last)):
787
+ matrix = _apply_tactile_scale(_decode_tactile_matrix(rows.get(frame)),
788
+ scale_factor)
789
  if matrix is None:
790
  blocks.append(bytes(pixels))
791
+ pairs.extend((lo, hi))
792
  continue
793
  pressure = np.maximum(matrix[2], 0.0)
794
  pressure = cv2.GaussianBlur(pressure, (3, 3), 0.8)
 
 
 
 
 
 
795
  scaled = np.clip((pressure - lo) / (hi - lo + 1e-6), 0.0, 1.0)
796
  blocks.append(np.rint(scaled * 255.0).astype(np.uint8).tobytes())
797
  pairs.extend((lo, hi))
processor/app/umi_slam_action.py ADDED
@@ -0,0 +1,269 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """UMI Gripper 动作派生 —— 从 SLAM 位姿反推 imitation learning 的 action。
2
+
3
+ UMI 硬件不录 action:采集端写入的 ``action`` 列是占位全零,导出时会被
4
+ ``lerobot_export._junk_columns`` 当零信息列剔除,导致数据集无法训练。
5
+ 官方 UMI 的做法同样是从 SLAM 位姿反推 —— 本模块实现这套口径。
6
+
7
+ action(7 维,当前夹爪局部坐标系):
8
+
9
+ Δp = R_tᵀ · (p_{t+1} − p_t) # 平移增量, 米
10
+ Δr = rotvec(R_tᵀ · R_{t+1}) # 旋转增量, 弧度
11
+ g = gripper_state[t+1, 0] / 100 # 夹爪开合, 0–1
12
+
13
+ 本模块是纯函数(不依赖 Web 框架与数据库),便于单测与跨格式复用。
14
+ """
15
+
16
+ from __future__ import annotations
17
+
18
+ import numpy as np
19
+
20
+ # 动作维度:Δp(3) + Δr(3) + 夹爪(1)
21
+ ACTION_DIM = 7
22
+
23
+
24
+ def quat_to_matrix(q) -> np.ndarray | None:
25
+ """四元数 ``(x, y, z, w)`` → 3×3 旋转矩阵。
26
+
27
+ 范数过小(零四元数)说明该帧没有有效姿态,返回 ``None`` 而不是抛异常 ——
28
+ 调用方据此跳过该帧。
29
+ """
30
+ x, y, z, w = (float(v) for v in np.asarray(q, dtype=float).ravel()[:4])
31
+ norm = (x * x + y * y + z * z + w * w) ** 0.5
32
+ if norm < 1e-12:
33
+ return None
34
+ x, y, z, w = x / norm, y / norm, z / norm, w / norm
35
+ return np.array([
36
+ [1 - 2 * (y * y + z * z), 2 * (x * y - z * w), 2 * (x * z + y * w)],
37
+ [2 * (x * y + z * w), 1 - 2 * (x * x + z * z), 2 * (y * z - x * w)],
38
+ [2 * (x * z - y * w), 2 * (y * z + x * w), 1 - 2 * (x * x + y * y)],
39
+ ])
40
+
41
+
42
+ def matrix_to_rotvec(rotation: np.ndarray) -> np.ndarray:
43
+ """3×3 旋转矩阵 → 旋转向量(轴×角,弧度)。
44
+
45
+ 近 180° 时 ``sin(角) → 0``,标准轴公式会除零并产生 NaN。手部大幅翻转
46
+ 是真实存在的姿态,一旦产生 NaN 会污染整列 action,因此单独走
47
+ ``(R + I)/2`` 的对角元求轴。
48
+ """
49
+ trace = (float(rotation[0, 0]) + float(rotation[1, 1])
50
+ + float(rotation[2, 2]) - 1.0) / 2.0
51
+ angle = float(np.arccos(max(-1.0, min(1.0, trace))))
52
+
53
+ if angle < 1e-9:
54
+ return np.zeros(3)
55
+
56
+ if np.pi - angle < 1e-6:
57
+ # 近 π:用 (R+I)/2 的列向量恢复转轴,再按对角元归一化。
58
+ symmetric = (rotation + np.eye(3)) / 2.0
59
+ diagonal = np.clip(np.diag(symmetric), 0.0, None)
60
+ index = int(np.argmax(diagonal))
61
+ if diagonal[index] < 1e-9:
62
+ return np.zeros(3)
63
+ axis = symmetric[:, index] / (diagonal[index] ** 0.5)
64
+ norm = float(np.linalg.norm(axis))
65
+ return axis / norm * angle if norm > 1e-9 else np.zeros(3)
66
+
67
+ axis = np.array([
68
+ float(rotation[2, 1]) - float(rotation[1, 2]),
69
+ float(rotation[0, 2]) - float(rotation[2, 0]),
70
+ float(rotation[1, 0]) - float(rotation[0, 1]),
71
+ ]) / (2.0 * np.sin(angle))
72
+ return axis * angle
73
+
74
+
75
+ def poses_from_trajectory(trajectories, timestamps,
76
+ frame_count: int) -> np.ndarray | None:
77
+ """从 ``observation.slam_trajectory`` 的样本缓冲重建逐帧位姿 ``(N, 7)``。
78
+
79
+ slam_trajectory 是采集端的高频(50Hz)样本缓冲,每格存
80
+ ``[t, x, y, z, qx, qy, qz, qw]`` 的整数倍。数据帧是 30fps,所以两个
81
+ 数据帧之间到达的样本会落在其中一格的格子里 —— 某一帧自己的格子为空,
82
+ 不代表那个时刻没有位姿,**相邻格子里往往就有**。
83
+
84
+ 因此正确做法是把所有格子的样本汇总,按时间戳逐帧取最近的:
85
+ 实测 30% 的「丢帧」里有 97% 能被真实样本覆盖,且单帧位移上限从
86
+ 113.9mm 降到 24.8mm(插值方案会横跨 SLAM 重定位跳变)。
87
+
88
+ 时间基映射:slam 时间戳与数据时间戳之间是**线性关系**(实测斜率
89
+ 1.02,采集端时钟略快)。用「每个非空格子的最后一组对应本帧时刻」
90
+ 的锚点做最小二乘拟合,不依赖外部时钟。
91
+
92
+ 返回 ``(frame_count, 7)``;样本不足返回 ``None``。
93
+ """
94
+ import numpy as np
95
+
96
+ if frame_count <= 0:
97
+ return None
98
+ stamp = np.asarray(timestamps, dtype=float).reshape(-1)[:frame_count]
99
+ if stamp.size < frame_count:
100
+ stamp = np.concatenate([stamp,
101
+ np.full(frame_count - stamp.size, np.nan)])
102
+
103
+ # 1) 汇总样本,并收集时间基锚点(非空格子的最后一组 ≈ 本帧时刻)
104
+ samples: list[tuple[float, list[float]]] = []
105
+ anchor_slam: list[float] = []
106
+ anchor_data: list[float] = []
107
+ has_anchor = ~np.isnan(stamp)
108
+ for index in range(min(frame_count, len(trajectories))):
109
+ values = np.asarray(trajectories[index], dtype=float).reshape(-1)
110
+ groups = values.size // 8
111
+ if groups <= 0:
112
+ continue
113
+ for g in range(groups):
114
+ chunk = values[g * 8:g * 8 + 8]
115
+ if not np.isfinite(chunk).all():
116
+ continue
117
+ samples.append((float(chunk[0]), [float(v) for v in chunk[1:8]]))
118
+ if has_anchor[index]:
119
+ anchor_slam.append(float(values[(groups - 1) * 8]))
120
+ anchor_data.append(float(stamp[index]))
121
+ if len(samples) < 2 or len(anchor_slam) < 2:
122
+ return None
123
+
124
+ # 2) 线性映射 slam 时间 → 数据时间:data = slope*slam + intercept
125
+ slam_t = np.array(anchor_slam, dtype=float)
126
+ data_t = np.array(anchor_data, dtype=float)
127
+ slope, intercept = np.polyfit(slam_t, data_t, 1)
128
+ if not np.isfinite(slope) or slope <= 0:
129
+ return None
130
+
131
+ samples.sort(key=lambda item: item[0])
132
+ sample_t = (np.array([item[0] for item in samples], dtype=float)
133
+ * slope + intercept)
134
+ sample_p = np.array([item[1] for item in samples], dtype=float)
135
+
136
+ # 3) 逐帧取时间上最近的样本。超过约一帧半间隔就认为真的没有数据,
137
+ # 标为缺失留给插值,避免把远处的样本硬贴到当前帧。
138
+ poses = np.zeros((frame_count, 7), dtype=float)
139
+ missing = np.zeros(frame_count, dtype=bool)
140
+ dt = np.nanmedian(np.diff(stamp)) if frame_count > 1 else 1.0 / 30.0
141
+ if not np.isfinite(dt) or dt <= 0:
142
+ dt = 1.0 / 30.0
143
+ tolerance = dt * 1.5
144
+ for index in range(frame_count):
145
+ if np.isnan(stamp[index]):
146
+ missing[index] = True
147
+ continue
148
+ j = int(np.abs(sample_t - stamp[index]).argmin())
149
+ if abs(sample_t[j] - stamp[index]) > tolerance:
150
+ missing[index] = True
151
+ continue
152
+ poses[index] = sample_p[j]
153
+
154
+ # 4) 真实样本覆盖不到的少数帧(实测约 3%)才插值兜底
155
+ valid = np.where(~missing)[0]
156
+ if len(valid) < 2:
157
+ return None
158
+ for index in np.where(missing)[0]:
159
+ lo = valid[valid < index]
160
+ hi = valid[valid > index]
161
+ if len(lo) and len(hi):
162
+ a, b = int(lo[-1]), int(hi[0])
163
+ w = (index - a) / (b - a)
164
+ poses[index] = (1 - w) * poses[a] + w * poses[b]
165
+ else:
166
+ poses[index] = poses[int(lo[-1])] if len(lo) else poses[int(hi[0])]
167
+
168
+ norms = np.linalg.norm(poses[:, 3:7], axis=1, keepdims=True)
169
+ poses[:, 3:7] = poses[:, 3:7] / np.maximum(norms, 1e-12)
170
+ return poses
171
+
172
+
173
+ def _gripper_fraction(states, frame_count: int) -> np.ndarray:
174
+ """取夹爪开合度 → 0–1。契约 ``[percent, gripped, raw]``,只用首维。
175
+
176
+ ``gripped`` 恒为 0、``raw`` 是带噪编码器计数,都不参与 action。
177
+ """
178
+ fraction = np.zeros(frame_count)
179
+ total = len(states) if states is not None else 0
180
+ for frame in range(frame_count):
181
+ raw = states[frame] if frame < total else None
182
+ if raw is None:
183
+ continue
184
+ values = np.asarray(raw, dtype=float).ravel()
185
+ if values.size:
186
+ fraction[frame] = float(values[0]) / 100.0
187
+ # 采集端开合度是 0–100;越界值(标定异常)夹到合法区间的边界,
188
+ # 避免动作维度出现 >1 的离群值把归一化统计带偏。
189
+ return np.clip(fraction, 0.0, 1.0)
190
+
191
+
192
+ def derive_action(trajectories, timestamps, gripper_states,
193
+ frame_count: int) -> dict[int, dict] | None:
194
+ """派生 UMI 口径的 action 与 proprioception。
195
+
196
+ 位姿**只从** ``observation.slam_trajectory`` 重建 —— 它是采集端的高频
197
+ 样本缓冲,丢帧时相邻格子里仍有真实样本(实测可覆盖 97% 的丢帧);
198
+ 而 ``observation.slam_pose`` 在同样位置直接写零,只能靠插值硬补,
199
+ 还会横跨 SLAM 重定位跳变(实测产生过 113.9mm 的假位移)。
200
+
201
+ 返回 ``{frame_index: {"action": [7], "observation.state": [4]}}``:
202
+
203
+ - ``action``(7 维):当前夹爪局部系的增量位姿 + 夹爪开合,见上文;
204
+ - ``observation.state``(4 维):``[Δx, Δy, Δz, gripper]``,位置为
205
+ **相对本集起点**的偏移。ACT 用单帧观测,没有 state 时模型无从得知
206
+ 夹爪当前在哪(只能看图像猜);而 SLAM 的世界原点是每集任意初始化的,
207
+ 给绝对坐标会把无关的世界位置当特征,故取相对量。
208
+
209
+ 输入按 ``frame_index`` 顺序排列。无可用样本(非 UMI 数据、或全程丢帧)
210
+ 返回 ``None``,调用方应跳过该集而非写入垃圾。
211
+
212
+ 末帧没有 ``t+1``,沿用倒数第二帧的增量 —— 与 LeRobot 的
213
+ ``next.done`` 语义一致(末帧只是终止标记,不承载新动作)。
214
+ """
215
+ if frame_count <= 0:
216
+ return None
217
+ cleaned = poses_from_trajectory(trajectories, timestamps, frame_count)
218
+ if cleaned is None:
219
+ return None
220
+ if len(cleaned) != frame_count:
221
+ # 位姿列与帧数不一致时以较短者为准,避免错位写入。
222
+ frame_count = min(frame_count, len(cleaned))
223
+
224
+ fraction = _gripper_fraction(gripper_states, frame_count)
225
+ # observation.state 的位置分量取**相对本集起点**的偏移。SLAM 的世界
226
+ # 原点是每次采集初始化时任意确定的,直接给绝对坐标会让模型把无关的
227
+ # 世界坐标当特征;相对起点才是可迁移的"相对自己出发点多远"。
228
+ origin = cleaned[0, :3].copy()
229
+ result: dict[int, dict] = {}
230
+
231
+ for frame in range(frame_count):
232
+ if frame < frame_count - 1:
233
+ current = quat_to_matrix(cleaned[frame, 3:7])
234
+ following = quat_to_matrix(cleaned[frame + 1, 3:7])
235
+ index = frame + 1
236
+ else:
237
+ # 末帧沿用倒数第二帧的位姿对,只取夹爪当前值。
238
+ if frame_count < 2:
239
+ continue
240
+ current = quat_to_matrix(cleaned[frame - 1, 3:7])
241
+ following = quat_to_matrix(cleaned[frame, 3:7])
242
+ index = frame
243
+ if current is None or following is None:
244
+ continue
245
+
246
+ if frame < frame_count - 1:
247
+ delta_position = current.T @ (cleaned[frame + 1, :3]
248
+ - cleaned[frame, :3])
249
+ else:
250
+ delta_position = current.T @ (cleaned[frame, :3]
251
+ - cleaned[frame - 1, :3])
252
+ delta_rotation = matrix_to_rotvec(current.T @ following)
253
+
254
+ action = np.concatenate([
255
+ delta_position, delta_rotation, [fraction[min(index, frame_count - 1)]],
256
+ ])
257
+ # 非有限值宁缺勿脏:该帧不写,由下游按缺失处理。
258
+ if not np.all(np.isfinite(action)):
259
+ continue
260
+ result[frame] = {
261
+ "action": [float(value) for value in action],
262
+ "observation.state": [
263
+ float(value) for value in np.concatenate([
264
+ cleaned[frame, :3] - origin,
265
+ [fraction[frame]],
266
+ ])
267
+ ],
268
+ }
269
+ return result or None
processor/tests/test_batch_fingerprint.py ADDED
@@ -0,0 +1,181 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """内容指纹:识别"换个包名重传同一份数据"。
2
+
3
+ 实测背景:采集端把已上传的录制重新推一遍时包名会变(本地去重标记丢失
4
+ 或包名规则变化),``session.py`` 的 incoming_name 判断不出来,同一份数据
5
+ 被追加成新 episode(UMIGripper_AI 出现 4 组字节级相同的批次)。这里的
6
+ 指纹必须与包名无关,且对"同一次录制的两份拷贝"稳定命中。
7
+ """
8
+
9
+ import json
10
+ import tempfile
11
+ import unittest
12
+ from pathlib import Path
13
+ from unittest import mock
14
+
15
+ import numpy as np
16
+ import pyarrow as pa
17
+ import pyarrow.parquet as pq
18
+
19
+ from app import batch_fingerprint
20
+
21
+
22
+ def _write_episode(root: Path, episode_index: int, video_bytes: bytes,
23
+ frames: int = 4) -> None:
24
+ """写一个最小 canonical 布局:一个视频 + 一个 data parquet。"""
25
+ videos = root / "videos" / "observation.images.gripper_rgb" / "chunk-000"
26
+ videos.mkdir(parents=True, exist_ok=True)
27
+ (videos / f"episode_{episode_index:06d}.mp4").write_bytes(video_bytes)
28
+
29
+ data = root / "data" / "chunk-000"
30
+ data.mkdir(parents=True, exist_ok=True)
31
+ table = pa.table({
32
+ "episode_index": pa.array([episode_index] * frames, pa.int64()),
33
+ "frame_index": pa.array(list(range(frames)), pa.int64()),
34
+ "observation.value": pa.array(
35
+ [[float(i), float(i) + 1.0] for i in range(frames)],
36
+ pa.list_(pa.float32()),
37
+ ),
38
+ })
39
+ pq.write_table(table, data / f"episode_{episode_index:06d}.parquet")
40
+
41
+
42
+ class FingerprintTests(unittest.TestCase):
43
+ def setUp(self):
44
+ self._tmp = tempfile.TemporaryDirectory()
45
+ self.root = Path(self._tmp.name)
46
+ # 索引落到临时目录,别碰真实 state/
47
+ self._index_patch = mock.patch.object(
48
+ batch_fingerprint, "INDEX_PATH", self.root / "fingerprints.json")
49
+ self._index_patch.start()
50
+
51
+ def tearDown(self):
52
+ self._index_patch.stop()
53
+ self._tmp.cleanup()
54
+
55
+ def test_same_recording_different_episode_index_matches(self):
56
+ """同一次录制的两份拷贝,只有 episode_index 不同 → 指纹相同。"""
57
+ a = self.root / "a"
58
+ b = self.root / "b"
59
+ _write_episode(a, 3, b"IDENTICAL-VIDEO-BYTES")
60
+ _write_episode(b, 4, b"IDENTICAL-VIDEO-BYTES")
61
+
62
+ self.assertIsNotNone(batch_fingerprint.compute_fingerprint(a))
63
+ self.assertEqual(
64
+ batch_fingerprint.compute_fingerprint(a),
65
+ batch_fingerprint.compute_fingerprint(b),
66
+ )
67
+
68
+ def test_different_video_differs(self):
69
+ a = self.root / "a"
70
+ b = self.root / "b"
71
+ _write_episode(a, 0, b"VIDEO-ONE")
72
+ _write_episode(b, 0, b"VIDEO-TWO")
73
+ self.assertNotEqual(
74
+ batch_fingerprint.compute_fingerprint(a),
75
+ batch_fingerprint.compute_fingerprint(b),
76
+ )
77
+
78
+ def test_different_frame_data_differs(self):
79
+ """视频相同但帧数据不同 → 不同指纹(不能只比视频)。"""
80
+ a = self.root / "a"
81
+ b = self.root / "b"
82
+ _write_episode(a, 0, b"SAME-VIDEO")
83
+ _write_episode(b, 0, b"SAME-VIDEO")
84
+ # 只改 b 的一帧数据
85
+ path = b / "data" / "chunk-000" / "episode_000000.parquet"
86
+ table = pq.read_table(path).to_pydict()
87
+ table["observation.value"] = [[9.0, 9.0] for _ in table["frame_index"]]
88
+ pq.write_table(pa.table(table), path)
89
+ self.assertNotEqual(
90
+ batch_fingerprint.compute_fingerprint(a),
91
+ batch_fingerprint.compute_fingerprint(b),
92
+ )
93
+
94
+ def test_missing_data_returns_none(self):
95
+ """没有 parquet 就算不出指纹 → None(调用方按新批次放行)。"""
96
+ empty = self.root / "empty"
97
+ empty.mkdir()
98
+ self.assertIsNone(batch_fingerprint.compute_fingerprint(empty))
99
+
100
+ def test_record_and_lookup_roundtrip(self):
101
+ fingerprint = "v:test=abc;d:def"
102
+ batch_fingerprint.record_fingerprint("UMIGripper_AI", fingerprint, "ep_000003")
103
+ self.assertEqual(
104
+ batch_fingerprint.lookup_duplicate(
105
+ "UMIGripper_AI", fingerprint, {"ep_000003"}),
106
+ "ep_000003",
107
+ )
108
+
109
+ def test_lookup_ignores_deleted_episode(self):
110
+ """索引指向的 episode 已删除 → 不判重(允许重新上传)。"""
111
+ fingerprint = "v:test=abc;d:def"
112
+ batch_fingerprint.record_fingerprint("UMIGripper_AI", fingerprint, "ep_000003")
113
+ self.assertIsNone(
114
+ batch_fingerprint.lookup_duplicate(
115
+ "UMIGripper_AI", fingerprint, {"ep_000004"}),
116
+ )
117
+
118
+ def test_lookup_none_fingerprint_is_safe(self):
119
+ self.assertIsNone(
120
+ batch_fingerprint.lookup_duplicate("p", None, {"a"}))
121
+
122
+ def test_record_drops_stale_entry_for_same_episode(self):
123
+ """同名重传换了内容:旧指纹不该继续指向这个 episode。"""
124
+ batch_fingerprint.record_fingerprint("p", "old-content", "ep_1")
125
+ batch_fingerprint.record_fingerprint("p", "new-content", "ep_1")
126
+ self.assertIsNone(
127
+ batch_fingerprint.lookup_duplicate("p", "old-content", {"ep_1"}))
128
+ self.assertEqual(
129
+ batch_fingerprint.lookup_duplicate("p", "new-content", {"ep_1"}),
130
+ "ep_1",
131
+ )
132
+
133
+ def test_forget_fingerprint(self):
134
+ batch_fingerprint.record_fingerprint("p", "abc", "ep_1")
135
+ batch_fingerprint.forget_fingerprint("ep_1")
136
+ self.assertIsNone(
137
+ batch_fingerprint.lookup_duplicate("p", "abc", {"ep_1"}))
138
+
139
+ def test_corrupt_index_is_tolerated(self):
140
+ batch_fingerprint.INDEX_PATH.write_text("{not json", encoding="utf-8")
141
+ self.assertIsNone(
142
+ batch_fingerprint.lookup_duplicate("p", "abc", {"ep_1"}))
143
+ batch_fingerprint.record_fingerprint("p", "abc", "ep_1")
144
+ self.assertEqual(
145
+ batch_fingerprint.lookup_duplicate("p", "abc", {"ep_1"}), "ep_1")
146
+
147
+ def test_index_is_valid_json_with_episode_map(self):
148
+ batch_fingerprint.record_fingerprint("proj", "fp1", "ep_1")
149
+ data = json.loads(batch_fingerprint.INDEX_PATH.read_text(encoding="utf-8"))
150
+ self.assertEqual(data["proj"]["fp1"], "ep_1")
151
+
152
+
153
+ class ParquetHashTests(unittest.TestCase):
154
+ def test_episode_index_column_is_excluded(self):
155
+ with tempfile.TemporaryDirectory() as tmp:
156
+ root = Path(tmp)
157
+ table = pa.table({
158
+ "episode_index": pa.array([7], pa.int64()),
159
+ "frame_index": pa.array([0], pa.int64()),
160
+ "v": pa.array([[1.0, 2.0]], pa.list_(pa.float32())),
161
+ })
162
+ one = root / "one.parquet"
163
+ pq.write_table(table, one)
164
+ other = root / "other.parquet"
165
+ pq.write_table(
166
+ table.set_column(0, "episode_index",
167
+ pa.array([99], pa.int64())), other)
168
+ self.assertEqual(
169
+ batch_fingerprint._parquet_content_hash(one),
170
+ batch_fingerprint._parquet_content_hash(other),
171
+ )
172
+
173
+ def test_unreadable_parquet_returns_none(self):
174
+ with tempfile.TemporaryDirectory() as tmp:
175
+ bad = Path(tmp) / "bad.parquet"
176
+ bad.write_bytes(b"not a parquet file")
177
+ self.assertIsNone(batch_fingerprint._parquet_content_hash(bad))
178
+
179
+
180
+ if __name__ == "__main__":
181
+ unittest.main()
processor/tests/test_device_task_progress.py ADDED
@@ -0,0 +1,181 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """采集端任务进度账本单测(app/device_progress.py)。
2
+
3
+ 覆盖:幂等重放、多设备聚合、增量水位、参数校验、未知任务拒绝(400 而非
4
+ 404 —— 采集端把 404 当"后端未实现"永久降级)、持久化与坏文件容错。
5
+
6
+ pytest tests/test_device_task_progress.py
7
+ python3 tests/test_device_task_progress.py # 无 pytest 也能跑
8
+ """
9
+ import json
10
+ import sys
11
+ from pathlib import Path
12
+
13
+ sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
14
+
15
+ from app import device_progress as dp # noqa: E402
16
+
17
+ TASK = "task-aaa"
18
+ OTHER = "task-bbb"
19
+ KNOWN = {TASK, OTHER}
20
+
21
+
22
+ def _report(tmp, increment, session_id, task=TASK, device="EGO_001"):
23
+ return dp.apply_report(
24
+ {"task_id": task, "device_name": device,
25
+ "session_id": session_id, "increment": increment},
26
+ KNOWN, path=tmp)
27
+
28
+
29
+ def test_first_report_sets_total(tmp_path):
30
+ f = tmp_path / "p.json"
31
+ out = _report(f, 1, "EGO_001:t:0")
32
+ assert out["applied"] is True
33
+ assert out["completed_count"] == 1
34
+ assert dp.reported_total(TASK, path=f) == 1
35
+
36
+
37
+ def test_replayed_watermark_is_idempotent(tmp_path):
38
+ """采集端崩溃重启后会重发同一水位 —— 不能重复计数。"""
39
+ f = tmp_path / "p.json"
40
+ first = _report(f, 12, "EGO_001:t:0")
41
+ again = _report(f, 12, "EGO_001:t:0")
42
+ assert first["completed_count"] == 12
43
+ assert again["applied"] is False
44
+ assert again["completed_count"] == 12
45
+ # 重放仍要回当前聚合数(采集端拿它当后端权威数)
46
+ assert isinstance(again["completed_count"], int)
47
+ assert not isinstance(again["completed_count"], bool)
48
+
49
+
50
+ def test_incremental_watermarks_accumulate(tmp_path):
51
+ f = tmp_path / "p.json"
52
+ _report(f, 12, "EGO_001:t:0")
53
+ out = _report(f, 1, "EGO_001:t:12")
54
+ assert out["completed_count"] == 13
55
+
56
+
57
+ def test_multiple_devices_aggregate(tmp_path):
58
+ f = tmp_path / "p.json"
59
+ _report(f, 3, "EGO_001:t:0", device="EGO_001")
60
+ _report(f, 2, "EGO_002:t:0", device="EGO_002")
61
+ out = _report(f, 1, "EGO_003:t:0", device="EGO_003")
62
+ assert out["completed_count"] == 6
63
+ # 同一水位键跨设备不冲突
64
+ assert dp.reported_total(TASK, path=f) == 6
65
+
66
+
67
+ def test_zero_increment_is_recorded_without_growth(tmp_path):
68
+ f = tmp_path / "p.json"
69
+ _report(f, 5, "EGO_001:t:0")
70
+ out = _report(f, 0, "EGO_001:t:5")
71
+ assert out["applied"] is True and out["completed_count"] == 5
72
+
73
+
74
+ def test_totals_only_contain_reported_tasks(tmp_path):
75
+ """没上报过的项目不能出现在这里 —— 调用方据此回退 session 计数。"""
76
+ f = tmp_path / "p.json"
77
+ _report(f, 1, "EGO_001:t:0")
78
+ totals = dp.reported_totals(path=f)
79
+ assert totals == {TASK: 1}
80
+ assert dp.reported_total(OTHER, path=f) is None
81
+
82
+
83
+ def test_validation_errors(tmp_path):
84
+ f = tmp_path / "p.json"
85
+ bad = [
86
+ ({}, "task_id"), # 空 body
87
+ ({"task_id": TASK, "session_id": "s"}, "device_name"),
88
+ ({"task_id": TASK, "device_name": "d"}, "session_id"),
89
+ ({"task_id": TASK, "device_name": "d", "session_id": "s",
90
+ "increment": -1}, "increment"),
91
+ ({"task_id": TASK, "device_name": "d", "session_id": "s",
92
+ "increment": "x"}, "increment"),
93
+ ({"task_id": TASK, "device_name": "d", "session_id": "s",
94
+ "increment": True}, "increment"),
95
+ ({"task_id": TASK, "device_name": "d", "session_id": "s",
96
+ "increment": dp.MAX_INCREMENT + 1}, "increment"),
97
+ ]
98
+ for body, needle in bad:
99
+ try:
100
+ dp.apply_report(body, KNOWN, path=f)
101
+ except ValueError as exc:
102
+ assert needle in str(exc), (body, str(exc))
103
+ else:
104
+ raise AssertionError(f"应当拒绝: {body}")
105
+ assert dp.reported_totals(path=f) == {} # 拒绝的请求不落账
106
+
107
+
108
+ def test_unknown_task_rejected_not_silently_accepted(tmp_path):
109
+ """未知任务要报错(路由转 400)。绝不能 404:采集端会永久降级上报。"""
110
+ f = tmp_path / "p.json"
111
+ try:
112
+ _report(f, 1, "EGO_001:t:0", task="no-such-task")
113
+ except ValueError as exc:
114
+ assert "unknown task_id" in str(exc)
115
+ else:
116
+ raise AssertionError("未知任务应当被拒绝")
117
+ # known_task_ids=None(不校验)时照常落账 —— 路由必须传已知集合
118
+ out = dp.apply_report({"task_id": "no-such-task", "device_name": "d",
119
+ "session_id": "s", "increment": 1}, None, path=f)
120
+ assert out["completed_count"] == 1
121
+
122
+
123
+ def test_count_prefers_report_over_sessions(tmp_path):
124
+ """核心口径:有上报就用录制数 —— 之后再上传多少条都不动它。"""
125
+ f = tmp_path / "p.json"
126
+ proj = {"id": TASK, "name": "UMIGripper_AI"}
127
+ _report(f, 12, "EGO_001:t:0")
128
+ reported = dp.reported_totals(path=f)
129
+ assert dp.resolve_count(proj, reported, sessions=16) == (12, "reported")
130
+ # 又上传了 3 条(项目下 session 变 19)→ 进度不变
131
+ assert dp.resolve_count(proj, reported, sessions=19) == (12, "reported")
132
+ # 录了一条 → 上报后 +1
133
+ _report(f, 1, "EGO_001:t:12")
134
+ assert dp.resolve_count(proj, dp.reported_totals(path=f),
135
+ sessions=19) == (13, "reported")
136
+
137
+
138
+ def test_count_falls_back_to_sessions_before_any_report(tmp_path):
139
+ """升级前就有数据的历史项目不归零:没有上报时仍按 session 数。"""
140
+ proj = {"id": TASK, "name": "UMIGripper_AI"}
141
+ assert dp.resolve_count(proj, {}, sessions=16) == (16, "sessions")
142
+ assert dp.needs_session_fallback([proj], {}) is True
143
+ assert dp.needs_session_fallback([proj], {TASK: 12}) is False
144
+
145
+
146
+ def test_persisted_across_reload(tmp_path):
147
+ f = tmp_path / "p.json"
148
+ _report(f, 2, "EGO_001:t:0")
149
+ raw = json.loads(f.read_text(encoding="utf-8"))
150
+ assert raw["tasks"][TASK]["devices"]["EGO_001"]["count"] == 2
151
+ # 重新读文件(模拟进程重启)后水位仍生效
152
+ again = _report(f, 2, "EGO_001:t:0")
153
+ assert again["applied"] is False and again["completed_count"] == 2
154
+
155
+
156
+ def test_corrupt_file_degrades_to_empty(tmp_path):
157
+ f = tmp_path / "p.json"
158
+ f.write_text("{ not json", encoding="utf-8")
159
+ assert dp.reported_totals(path=f) == {}
160
+ out = _report(f, 1, "EGO_001:t:0")
161
+ assert out["completed_count"] == 1
162
+
163
+
164
+ if __name__ == "__main__":
165
+ import tempfile
166
+ import traceback
167
+
168
+ failed = 0
169
+ for name, fn in sorted(globals().items()):
170
+ if not name.startswith("test_") or not callable(fn):
171
+ continue
172
+ with tempfile.TemporaryDirectory() as d:
173
+ try:
174
+ fn(Path(d))
175
+ print(f" PASS: {name}")
176
+ except Exception:
177
+ failed += 1
178
+ print(f" FAIL: {name}")
179
+ traceback.print_exc()
180
+ print("FAIL" if failed else "PASS: device_task_progress 全部通过")
181
+ sys.exit(1 if failed else 0)
processor/tests/test_lerobot_export.py ADDED
@@ -0,0 +1,105 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import unittest
2
+
3
+ import numpy as np
4
+
5
+ from app.lerobot_export import _ensure_slam_pose, _ensure_observation_state
6
+ from app.umi_slam_action import poses_from_trajectory
7
+
8
+
9
+ def _sample(t: float, x: float, z: float) -> list[float]:
10
+ """一组 slam_trajectory 样本:[t, x, y, z, qx, qy, qz, qw]。"""
11
+ return [float(t), float(x), 0.0, float(z), 0.0, 0.0, 0.0, 1.0]
12
+
13
+
14
+ def _rows(episode: int, count: int, *, t0: float,
15
+ slam_offset: float = 0.0) -> list[dict]:
16
+ """造一集 rows:每帧两个 slam 样本,位姿随帧号线性推进。"""
17
+ rows = []
18
+ for index in range(count):
19
+ stamp = t0 + index / 30.0
20
+ rows.append({
21
+ "episode_index": episode,
22
+ "timestamp": stamp,
23
+ "observation.slam_trajectory": [
24
+ _sample(stamp - slam_offset, index * 0.01, index * 0.002),
25
+ _sample(stamp - slam_offset + 0.01,
26
+ index * 0.01 + 0.001, index * 0.002),
27
+ ],
28
+ })
29
+ return rows
30
+
31
+
32
+ class EnsureSlamPoseTests(unittest.TestCase):
33
+ def test_derives_column_and_matches_direct_reconstruction(self):
34
+ """整列缺失时补出 slam_pose,且值与单独调用重建函数逐值一致。"""
35
+ rows = _rows(0, 40, t0=100.0)
36
+ self.assertEqual(_ensure_slam_pose(rows), "observation.slam_pose")
37
+
38
+ self.assertTrue(all(len(row["observation.slam_pose"]) == 7
39
+ for row in rows))
40
+ expected = poses_from_trajectory(
41
+ [row["observation.slam_trajectory"] for row in rows],
42
+ np.array([row["timestamp"] for row in rows]), len(rows))
43
+ actual = np.array([row["observation.slam_pose"] for row in rows])
44
+ self.assertTrue(np.allclose(actual, expected, atol=1e-12))
45
+
46
+ def test_episodes_are_fitted_independently(self):
47
+ """时间基映射必须每集单独拟合,否则跨集求差会把原点差异当运动。
48
+
49
+ 两集的 slam 时钟相差一个很大的偏移量;若混在一起拟合,时间轴被拉
50
+ 成两段,重建值会偏离该集单独重建的结果。
51
+ """
52
+ first = _rows(0, 30, t0=100.0, slam_offset=0.0)
53
+ second = _rows(1, 30, t0=500.0, slam_offset=400.0)
54
+ rows = first + second
55
+ self.assertEqual(_ensure_slam_pose(rows), "observation.slam_pose")
56
+
57
+ for group, offset in ((first, 0.0), (second, 400.0)):
58
+ expected = poses_from_trajectory(
59
+ [row["observation.slam_trajectory"] for row in group],
60
+ np.array([row["timestamp"] for row in group]), len(group),
61
+ )
62
+ actual = np.array([row["observation.slam_pose"] for row in group])
63
+ self.assertTrue(np.allclose(actual, expected, atol=1e-12))
64
+
65
+ def test_existing_column_is_left_untouched(self):
66
+ """旧批两列都有 —— 重导时不得覆盖采集端原始 slam_pose。"""
67
+ rows = _rows(0, 10, t0=100.0)
68
+ original = [9.0] * 7
69
+ for row in rows:
70
+ row["observation.slam_pose"] = list(original)
71
+ self.assertIsNone(_ensure_slam_pose(rows))
72
+ self.assertTrue(all(row["observation.slam_pose"] == original
73
+ for row in rows))
74
+
75
+ def test_skips_batches_without_trajectory(self):
76
+ """手套 / D435 批次没有 slam_trajectory —— 不得凭空写入该列。"""
77
+ rows = [{"episode_index": 0, "timestamp": 0.0,
78
+ "observation.tactile.left": [0.0] * 256}]
79
+ self.assertIsNone(_ensure_slam_pose(rows))
80
+ self.assertNotIn("observation.slam_pose", rows[0])
81
+
82
+ def test_missing_timestamp_does_not_break_reconstruction(self):
83
+ """时间戳缺帧(None)不能把 np.asarray 推断成 object 而抛异常。"""
84
+ rows = _rows(0, 20, t0=200.0)
85
+ rows[5]["timestamp"] = None
86
+ self.assertEqual(_ensure_slam_pose(rows), "observation.slam_pose")
87
+ self.assertTrue(all(np.all(np.isfinite(row["observation.slam_pose"]))
88
+ for row in rows))
89
+
90
+ def test_derived_pose_feeds_observation_state(self):
91
+ """串联顺序:补出的 slam_pose 要能继续供 state 派生使用。
92
+
93
+ 新批里 ep72/ep73 两列都缺,只补 slam_pose 而不让 state 跟上,导出
94
+ 产物仍然缺 observation.state,ACT 依旧 KeyError。
95
+ """
96
+ rows = _rows(0, 25, t0=300.0)
97
+ _ensure_slam_pose(rows)
98
+ self.assertEqual(_ensure_observation_state(rows), "observation.state")
99
+ self.assertTrue(all(len(row["observation.state"]) == 4 for row in rows))
100
+ # 位置分量是本帧相对本集第一帧的偏移 —— 首帧必须为原点
101
+ self.assertTrue(np.allclose(rows[0]["observation.state"][:3], 0.0))
102
+
103
+
104
+ if __name__ == "__main__":
105
+ unittest.main()
processor/tests/test_tactile_decode.py CHANGED
@@ -1,8 +1,16 @@
 
 
1
  import unittest
 
2
 
3
  import numpy as np
4
 
5
- from app.routes.video import _decode_tactile_matrix
 
 
 
 
 
6
 
7
 
8
  class TactileDecodeTests(unittest.TestCase):
@@ -23,5 +31,142 @@ class TactileDecodeTests(unittest.TestCase):
23
  np.testing.assert_array_equal(decoded.transpose(1, 2, 0), matrix)
24
 
25
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
26
  if __name__ == "__main__":
27
  unittest.main()
 
1
+ import json
2
+ import tempfile
3
  import unittest
4
+ from pathlib import Path
5
 
6
  import numpy as np
7
 
8
+ from app.routes.video import (
9
+ _apply_tactile_scale,
10
+ _compute_tactile_display_range,
11
+ _decode_tactile_matrix,
12
+ _tactile_scale_factor,
13
+ )
14
 
15
 
16
  class TactileDecodeTests(unittest.TestCase):
 
31
  np.testing.assert_array_equal(decoded.transpose(1, 2, 0), matrix)
32
 
33
 
34
+ class TactileScaleTests(unittest.TestCase):
35
+ """量化整数 → mN 的比例因子必须取自元数据,不能硬编码。
36
+
37
+ 新采集端按 ``scale: 100`` 存储(units 仍是 mN);旧数据无该字段且
38
+ 本身就是 mN。硬编码 100 会把旧数据错误缩小 100 倍。
39
+ """
40
+
41
+ def _dataset(self, root: Path, scale) -> Path:
42
+ (root / "meta").mkdir(parents=True)
43
+ (root / "data" / "chunk-000").mkdir(parents=True)
44
+ feature = {"dtype": "int16", "shape": [250, 250, 3],
45
+ "units": ["mN", "mN", "mN"]}
46
+ if scale is not None:
47
+ feature["scale"] = scale
48
+ (root / "meta" / "info.json").write_text(json.dumps({
49
+ "features": {"observation.gripper_left_force_matrix": feature},
50
+ }), encoding="utf-8")
51
+ return root / "data" / "chunk-000" / "episode_000000.parquet"
52
+
53
+ def test_reads_declared_scale(self):
54
+ with tempfile.TemporaryDirectory() as temp:
55
+ path = self._dataset(Path(temp), 100)
56
+ self.assertEqual(_tactile_scale_factor(path), 100.0)
57
+
58
+ def test_defaults_to_one_when_declared_or_missing(self):
59
+ with tempfile.TemporaryDirectory() as temp:
60
+ path = self._dataset(Path(temp), None)
61
+ self.assertEqual(_tactile_scale_factor(path), 1.0)
62
+
63
+ with tempfile.TemporaryDirectory() as temp:
64
+ path = self._dataset(Path(temp), 0)
65
+ self.assertEqual(_tactile_scale_factor(path), 1.0)
66
+
67
+ def test_missing_metadata_is_not_an_error(self):
68
+ with tempfile.TemporaryDirectory() as temp:
69
+ path = Path(temp) / "data" / "chunk-000" / "episode_000000.parquet"
70
+ path.parent.mkdir(parents=True)
71
+ self.assertEqual(_tactile_scale_factor(path), 1.0)
72
+
73
+ def test_old_data_is_left_untouched(self):
74
+ """scale=1 时不得改变数值 —— 旧数据必须保持原样。"""
75
+ matrix = np.full((3, 250, 250), 5.0, dtype=np.float32)
76
+ scaled = _apply_tactile_scale(matrix, 1.0)
77
+ np.testing.assert_array_equal(scaled, matrix)
78
+
79
+ def test_new_data_is_divided(self):
80
+ matrix = np.full((3, 250, 250), 250.0, dtype=np.float32)
81
+ scaled = _apply_tactile_scale(matrix, 100.0)
82
+ np.testing.assert_allclose(scaled, 2.5, atol=1e-6)
83
+
84
+ def test_none_matrix_is_passed_through(self):
85
+ self.assertIsNone(_apply_tactile_scale(None, 100.0))
86
+
87
+
88
+ def _write_episode(path: Path, matrices) -> Path:
89
+ """Write an episode whose row-diff column decodes back to ``matrices``."""
90
+ import pyarrow as pa
91
+ import pyarrow.parquet as pq
92
+
93
+ flattened = []
94
+ for matrix in matrices:
95
+ rows = np.asarray(matrix, dtype=np.int16).reshape(250, 750)
96
+ diff = np.empty_like(rows)
97
+ diff[:, 0] = rows[:, 0]
98
+ diff[:, 1:] = rows[:, 1:] - rows[:, :-1]
99
+ flattened.append(diff.reshape(-1))
100
+ table = pa.table({
101
+ "frame_index": pa.array(range(len(matrices)), type=pa.int64()),
102
+ "observation.gripper_left_force_matrix":
103
+ pa.array(flattened, type=pa.list_(pa.int16())),
104
+ })
105
+ path.parent.mkdir(parents=True, exist_ok=True)
106
+ pq.write_table(table, str(path))
107
+ return path
108
+
109
+
110
+ def _episode_frames(quiet=9, contact=80):
111
+ """A quiet episode (noise only) with one frame carrying real contact."""
112
+ rng = np.random.default_rng(20260911)
113
+ matrices = []
114
+ for _ in range(quiet):
115
+ frame = np.zeros((250, 250, 3), dtype=np.int16)
116
+ frame[:, :, 2] = rng.integers(0, 3, (250, 250))
117
+ matrices.append(frame)
118
+ loaded = np.zeros((250, 250, 3), dtype=np.int16)
119
+ loaded[100:130, 100:130, 2] = contact
120
+ matrices.append(loaded)
121
+ return matrices
122
+
123
+
124
+ class TactileDisplayRangeTests(unittest.TestCase):
125
+ """色阶必须取自整集,不能逐帧自比。
126
+
127
+ 没有接触的帧里唯一超过噪声门限的就是噪声本身;逐帧归一化会把这份噪声
128
+ 当成满量程,每个噪点都被拉成"接触点"。整集取色阶后安静帧保持背景色。
129
+ """
130
+
131
+ def _episode(self, temp: str):
132
+ path = _write_episode(
133
+ Path(temp) / "data" / "chunk-000" / "episode_000000.parquet",
134
+ _episode_frames())
135
+ return path
136
+
137
+ def test_range_is_driven_by_the_contact_not_the_noise(self):
138
+ with tempfile.TemporaryDirectory() as temp:
139
+ path = self._episode(temp)
140
+ lo, hi = _compute_tactile_display_range(
141
+ path, "observation.gripper_left_force_matrix")
142
+ self.assertGreaterEqual(hi, 80.0)
143
+ self.assertLess(lo, hi)
144
+
145
+ def test_quiet_frame_stays_background(self):
146
+ """整集色阶下,无接触的帧几乎全黑;旧逐帧逻辑在该帧给出 174 级亮度。"""
147
+ from app.routes.video import _tactile_range_payload
148
+
149
+ with tempfile.TemporaryDirectory() as temp:
150
+ path = self._episode(temp)
151
+ payload = _tactile_range_payload(path, 0, 10, "left")
152
+ self.assertIsNotNone(payload)
153
+ pixels = 250 * 250
154
+ blocks = np.frombuffer(payload[:10 * pixels], dtype=np.uint8)
155
+ quiet = blocks[:9 * pixels].reshape(9, pixels)
156
+ loud = blocks[9 * pixels:].reshape(pixels)
157
+ self.assertLess(float((quiet >= 128).mean()), 0.01)
158
+ self.assertGreater(int(loud.max()), 150)
159
+
160
+ def test_flat_episode_is_drawable(self):
161
+ with tempfile.TemporaryDirectory() as temp:
162
+ blank = np.zeros((250, 250, 3), dtype=np.int16)
163
+ path = _write_episode(
164
+ Path(temp) / "data" / "chunk-000" / "episode_000000.parquet",
165
+ [blank, blank])
166
+ lo, hi = _compute_tactile_display_range(
167
+ path, "observation.gripper_left_force_matrix")
168
+ self.assertLess(lo, hi)
169
+
170
+
171
  if __name__ == "__main__":
172
  unittest.main()
processor/tests/test_umi_slam_action.py ADDED
@@ -0,0 +1,369 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import math
2
+ import tempfile
3
+ import unittest
4
+ from pathlib import Path
5
+
6
+ import numpy as np
7
+ import pyarrow as pa
8
+ import pyarrow.parquet as pq
9
+
10
+ from app.processing import JobContext, ModuleSkip
11
+ from app.processing.modules.umi_slam_action import UmiSlamActionModule
12
+ from app.umi_slam_action import (
13
+ derive_action,
14
+ matrix_to_rotvec,
15
+ quat_to_matrix,
16
+ )
17
+
18
+
19
+ def _yaw_quat(angle: float) -> list[float]:
20
+ """绕 Z 轴 angle 弧度的四元数 (x, y, z, w)。"""
21
+ return [0.0, 0.0, math.sin(angle / 2.0), math.cos(angle / 2.0)]
22
+
23
+
24
+ def _pose(x, y, z, quat=None) -> list[float]:
25
+ return [float(x), float(y), float(z)] + (quat or [0.0, 0.0, 0.0, 1.0])
26
+
27
+
28
+ class RotationMathTests(unittest.TestCase):
29
+ def test_identity_and_quarter_turn(self):
30
+ # 单位四元数 → 单位阵 → 零旋转向量
31
+ self.assertTrue(np.allclose(matrix_to_rotvec(quat_to_matrix([0, 0, 0, 1])),
32
+ np.zeros(3), atol=1e-9))
33
+
34
+ # 绕 X 轴 90°
35
+ q = [math.sin(math.pi / 4), 0.0, 0.0, math.cos(math.pi / 4)]
36
+ self.assertTrue(np.allclose(matrix_to_rotvec(quat_to_matrix(q)),
37
+ [math.pi / 2, 0.0, 0.0], atol=1e-6))
38
+
39
+ def test_near_180_degrees_is_finite(self):
40
+ """近 180° 时 sin(角)→0,标准轴公式会除零 —— 必须走退化分支。"""
41
+ for angle in (math.pi, math.pi - 1e-7, math.pi - 1e-4):
42
+ q = [0.0, 0.0, math.sin(angle / 2.0), math.cos(angle / 2.0)]
43
+ rotvec = matrix_to_rotvec(quat_to_matrix(q))
44
+ self.assertTrue(np.all(np.isfinite(rotvec)),
45
+ f"angle={angle} produced non-finite {rotvec}")
46
+ self.assertAlmostEqual(float(np.linalg.norm(rotvec)), angle, places=5)
47
+
48
+ def test_zero_quaternion_returns_none(self):
49
+ self.assertIsNone(quat_to_matrix([0.0, 0.0, 0.0, 0.0]))
50
+
51
+
52
+ def _traj(poses, dt=1/30, offset=1000.0):
53
+ """把逐帧位姿包成 slam_trajectory 的格子。
54
+
55
+ 每格是 ``[slam_t, x, y, z, qx, qy, qz, qw]`` 的整数倍;slam 时间戳与
56
+ 数据时间戳差一个常数偏移(采集端时钟独立)。
57
+ """
58
+ return [[ (i * dt + offset) ] + list(pose) for i, pose in enumerate(poses)]
59
+
60
+
61
+ def _stamps(n, dt=1/30):
62
+ return [i * dt for i in range(n)]
63
+
64
+
65
+ class DeriveActionTests(unittest.TestCase):
66
+ def test_pure_translation_matches_analytic_delta(self):
67
+ """沿 +X 每帧 10mm、姿态恒为身份 → Δp 应为 [0.01, 0, 0]。"""
68
+ poses = [_pose(0.01 * i, 0, 0) for i in range(6)]
69
+ out = derive_action(_traj(poses), _stamps(6), [[0.0, 0, 0]] * 6, 6)
70
+ self.assertIsNotNone(out)
71
+ for frame in range(5):
72
+ self.assertTrue(np.allclose(out[frame]["action"][:3], [0.01, 0, 0],
73
+ atol=1e-9))
74
+ self.assertTrue(np.allclose(out[frame]["action"][3:6], [0, 0, 0],
75
+ atol=1e-9))
76
+
77
+ def test_local_frame_delta_follows_current_orientation(self):
78
+ """世界系位移须换算到**当前**夹爪局部系,而不是原样透传。"""
79
+ step = 0.02
80
+ quat = [0.0, 0.0, math.sin(-math.pi / 4), math.cos(-math.pi / 4)]
81
+ poses = [_pose(0.0, -step * frame, 0.0, quat) for frame in range(5)]
82
+ out = derive_action(_traj(poses), _stamps(5), None, 5)
83
+ self.assertTrue(np.allclose(out[0]["action"][:3], [step, 0, 0],
84
+ atol=1e-6))
85
+
86
+ def test_world_delta_is_not_passed_through_verbatim(self):
87
+ step = 0.02
88
+ quat = [0.0, 0.0, math.sin(-math.pi / 4), math.cos(-math.pi / 4)]
89
+ poses = [_pose(0.0, -step * frame, 0.0, quat) for frame in range(3)]
90
+ out = derive_action(_traj(poses), _stamps(3), None, 3)
91
+ self.assertFalse(np.allclose(out[0]["action"][:3], [0, -step, 0],
92
+ atol=1e-6))
93
+
94
+ def test_pure_rotation_matches_analytic_delta(self):
95
+ angle = 0.1
96
+ poses = [_pose(0, 0, 0, _yaw_quat(angle * i)) for i in range(5)]
97
+ out = derive_action(_traj(poses), _stamps(5), None, 5)
98
+ self.assertTrue(np.allclose(out[0]["action"][3:6], [0, 0, angle],
99
+ atol=1e-6))
100
+
101
+ def test_gripper_dimension_tracks_open_fraction(self):
102
+ poses = [_pose(0.01 * i, 0, 0) for i in range(4)]
103
+ gripper = [[0.0, 0, 0], [50.0, 0, 0], [100.0, 0, 0], [100.0, 0, 0]]
104
+ out = derive_action(_traj(poses), _stamps(4), gripper, 4)
105
+ self.assertAlmostEqual(out[0]["action"][6], 0.5, places=6)
106
+ self.assertAlmostEqual(out[1]["action"][6], 1.0, places=6)
107
+
108
+ def test_gripper_is_clipped_to_unit_range(self):
109
+ poses = [_pose(0.01 * i, 0, 0) for i in range(3)]
110
+ out = derive_action(_traj(poses), _stamps(3), [[500.0, 0, 0]] * 3, 3)
111
+ self.assertLessEqual(out[0]["action"][6], 1.0)
112
+ self.assertGreaterEqual(out[0]["action"][6], 0.0)
113
+
114
+ def test_last_frame_reuses_previous_delta(self):
115
+ poses = [_pose(0.01 * i, 0, 0) for i in range(5)]
116
+ out = derive_action(_traj(poses), _stamps(5), None, 5)
117
+ self.assertIn(4, out)
118
+ self.assertTrue(np.allclose(out[4]["action"][:3], out[3]["action"][:3],
119
+ atol=1e-9))
120
+
121
+ def test_action_dimension_is_seven(self):
122
+ poses = [_pose(0.01 * i, 0, 0) for i in range(5)]
123
+ out = derive_action(_traj(poses), _stamps(5), None, 5)
124
+ self.assertEqual(len(out[0]["action"]), 7)
125
+
126
+ def test_all_finite_on_mixed_input(self):
127
+ poses = [_pose(0.01 * i, 0, 0) for i in range(6)]
128
+ traj = _traj(poses)
129
+ traj[3] = [] # 该格没有样本
130
+ out = derive_action(traj, _stamps(6), None, 6)
131
+ values = np.array([out[i]["action"] for i in sorted(out)])
132
+ self.assertTrue(np.all(np.isfinite(values)))
133
+
134
+ def test_empty_and_degenerate_inputs(self):
135
+ self.assertIsNone(derive_action([], [], None, 0))
136
+ self.assertIsNone(derive_action([[]] * 3, _stamps(3), None, 3))
137
+ self.assertIsNone(derive_action(_traj([_pose(0, 0, 0)]), _stamps(1),
138
+ None, 1))
139
+
140
+ def test_sample_from_neighbour_cell_recovers_empty_frame(self):
141
+ """空帧若能从相邻格子的样本里找到位姿,就不该被当作丢帧。
142
+
143
+ 这是换用 slam_trajectory 的核心收益:采集端 50Hz、数据帧 30fps,
144
+ 两帧之间到达的样本落在其中一格,所以「本帧格子为空」不代表那一刻
145
+ 没有位姿。
146
+ """
147
+ # 真实形态:绝大多数帧的格子只装属于自己的那一个样本,个别格子会
148
+ # 多装一个「属于下一帧窗口」的样本。帧 5 自己的格子为空,但帧 4 的
149
+ # 格子里存着它 —— 这正是采集端 50Hz 快于数据 30fps 的结果。
150
+ dt = 1 / 30
151
+ n = 10
152
+ traj = []
153
+ for i in range(n):
154
+ if i == 5:
155
+ traj.append([]) # 本帧格子空
156
+ continue
157
+ cell = [1000.0 + i * dt] + _pose(0.01 * i, 0, 0)
158
+ if i == 4:
159
+ # 帧 5 的样本溢到了上一个格子里(它比本帧时刻晚一点)
160
+ cell += [1000.0 + 5 * dt] + _pose(0.05, 0, 0)
161
+ traj.append(cell)
162
+
163
+ out = derive_action(traj, _stamps(n), None, n)
164
+ self.assertIsNotNone(out)
165
+ self.assertIn(5, out, "空帧应能从相邻格子的样本恢复")
166
+ # 帧 5 用的应是那个真实样本(0.05),而不是帧 4/6 的插值中点
167
+ self.assertAlmostEqual(out[5]["observation.state"][0], 0.05, places=5)
168
+
169
+ def test_observation_state_is_relative_to_episode_start(self):
170
+ poses = [_pose(1.0 + 0.01 * i, 2.0, 3.0) for i in range(5)]
171
+ out = derive_action(_traj(poses), _stamps(5), [[50.0, 0, 0]] * 5, 5)
172
+ # 位置分量是相对本集首帧的偏移 → 首帧为 0
173
+ self.assertTrue(np.allclose(out[0]["observation.state"][:3],
174
+ [0.0, 0.0, 0.0], atol=1e-9))
175
+ self.assertAlmostEqual(out[3]["observation.state"][0], 0.03, places=6)
176
+ self.assertAlmostEqual(out[3]["observation.state"][3], 0.5, places=6)
177
+
178
+
179
+ class UmiSlamActionModuleTests(unittest.TestCase):
180
+ def _context(self, root: Path) -> JobContext:
181
+ return JobContext(
182
+ node_id="umi_slam_action",
183
+ node_type="umi_slam_action",
184
+ config={},
185
+ job={},
186
+ input_root=root,
187
+ output_root=root / "outputs",
188
+ incoming={},
189
+ progress=lambda _value: None,
190
+ node_states={},
191
+ )
192
+
193
+ def _write_episode(self, root: Path, poses, gripper=None,
194
+ extra_columns: dict | None = None) -> Path:
195
+ """写入一份 canonical parquet。
196
+
197
+ 位姿写成 ``observation.slam_trajectory`` 的格子格式(每格 8 的整数
198
+ 倍)并配 ``timestamp`` 列 —— 模块只从这个缓冲重建位姿。
199
+ """
200
+ (root / "data" / "chunk-000").mkdir(parents=True, exist_ok=True)
201
+ stamps = _stamps(len(poses))
202
+ columns = {
203
+ "frame_index": list(range(len(poses))),
204
+ "timestamp": stamps,
205
+ "observation.slam_trajectory": _traj(poses),
206
+ }
207
+ if gripper is not None:
208
+ columns["observation.gripper_state"] = gripper
209
+ for name, values in (extra_columns or {}).items():
210
+ columns[name] = values
211
+ path = root / "data" / "chunk-000" / "episode_000000.parquet"
212
+ pq.write_table(pa.table(columns), path)
213
+ return path
214
+
215
+ def test_derives_varying_action_and_reports_metadata(self):
216
+ with tempfile.TemporaryDirectory() as temp:
217
+ root = Path(temp)
218
+ # 用非等距轨迹:等距平移的每帧增量恒等,会被导出端的恒定列
219
+ # 过滤剔除,验证不出"动作确实在变化"这一点。
220
+ poses = [_pose(0.01 * i + 0.004 * (i % 3), 0, 0) for i in range(20)]
221
+ self._write_episode(root, poses, [[50.0, 0, 0]] * 20)
222
+
223
+ outputs = UmiSlamActionModule().run(self._context(root))
224
+
225
+ self.assertIn("action", outputs)
226
+ ref = outputs["action"]
227
+ self.assertEqual(ref.metadata["shape"], [7])
228
+ self.assertEqual(ref.metadata["frames_total"], 20)
229
+ self.assertEqual(len(ref.metadata["names"]), 7)
230
+
231
+ import pandas as pd
232
+ frame = pd.read_parquet(root / "outputs" / "umi_slam_action" / "umi_slam_action.parquet")
233
+ values = np.array(frame["action"].tolist())
234
+ self.assertEqual(values.shape[1], 7)
235
+ # 必须是真正变化的动作,否则导出的恒定列过滤会把它删掉
236
+ self.assertGreater(len(np.unique(values[:, 0])), 1)
237
+
238
+ def test_output_is_float32(self):
239
+ """LeRobot 的 action 声明是 float32;写 double 会造成声明与 schema
240
+ 不一致(官方加载器按声明 cast)。"""
241
+ import pyarrow.parquet as pq
242
+ with tempfile.TemporaryDirectory() as temp:
243
+ root = Path(temp)
244
+ poses = [_pose(0.01 * i, 0, 0) for i in range(10)]
245
+ self._write_episode(root, poses, [[50.0, 0, 0]] * 10)
246
+ UmiSlamActionModule().run(self._context(root))
247
+
248
+ path = root / "outputs" / "umi_slam_action" / "umi_slam_action.parquet"
249
+ field_type = pq.read_schema(path).field("action").type
250
+ self.assertIn("float", str(field_type))
251
+ self.assertNotIn("double", str(field_type))
252
+
253
+ def test_skips_batch_without_slam_pose(self):
254
+ """非 UMI 批次(如 D435 手套)必须跳过,不能写坏数据。"""
255
+ with tempfile.TemporaryDirectory() as temp:
256
+ root = Path(temp)
257
+ self._write_episode(root, [_pose(0, 0, 0)], None,
258
+ extra_columns={"observation.glove": [[1.0, 2.0]]})
259
+ # 覆盖掉 slam 列,模拟没有 SLAM 的设备
260
+ (root / "data" / "chunk-000" / "episode_000000.parquet").unlink()
261
+ pq.write_table(pa.table({
262
+ "frame_index": [0, 1],
263
+ "observation.glove": [[1.0, 2.0], [3.0, 4.0]],
264
+ }), root / "data" / "chunk-000" / "episode_000000.parquet")
265
+
266
+ with self.assertRaises(ModuleSkip):
267
+ UmiSlamActionModule().run(self._context(root))
268
+
269
+ def test_retries_transient_read_failure(self):
270
+ """上传/处理并发时 parquet 可能瞬时不可读,不能一次失败就放弃。
271
+
272
+ 实测 ep52/ep71 都因此被误判成「列不可读」跳过,几秒后同一文件完全
273
+ 正常。这里注入两次失败,验证第三次成功读到时模块照常产出。
274
+ """
275
+ from unittest.mock import patch
276
+
277
+ import pandas as pd
278
+
279
+ with tempfile.TemporaryDirectory() as temp:
280
+ root = Path(temp)
281
+ poses = [_pose(0.01 * i + 0.004 * (i % 3), 0, 0) for i in range(12)]
282
+ self._write_episode(root, poses, [[50.0, 0, 0]] * 12)
283
+
284
+ original = pd.read_parquet
285
+ state = {"calls": 0}
286
+
287
+ def flaky(*args, **kwargs):
288
+ state["calls"] += 1
289
+ if state["calls"] <= 2:
290
+ raise OSError("file is being written")
291
+ return original(*args, **kwargs)
292
+
293
+ with patch.object(pd, "read_parquet", flaky), \
294
+ patch("time.sleep", lambda _s: None):
295
+ outputs = UmiSlamActionModule().run(self._context(root))
296
+
297
+ self.assertGreaterEqual(state["calls"], 3, "应当重试到成功")
298
+ self.assertIn("action", outputs)
299
+
300
+ def test_skip_message_carries_the_real_exception(self):
301
+ """笼统提示无法区分竞态与设备类型不符,必须带上真实异常。"""
302
+ from unittest.mock import patch
303
+
304
+ import pandas as pd
305
+
306
+ with tempfile.TemporaryDirectory() as temp:
307
+ root = Path(temp)
308
+ self._write_episode(root, [_pose(0.01 * i, 0, 0) for i in range(8)],
309
+ [[50.0, 0, 0]] * 8)
310
+
311
+ def always_fail(*args, **kwargs):
312
+ raise OSError("mount is stale")
313
+
314
+ with patch.object(pd, "read_parquet", always_fail), \
315
+ patch("time.sleep", lambda _s: None):
316
+ with self.assertRaises(ModuleSkip) as caught:
317
+ UmiSlamActionModule().run(self._context(root))
318
+
319
+ message = str(caught.exception)
320
+ self.assertIn("OSError", message)
321
+ self.assertIn("stale", message)
322
+
323
+ def test_skips_when_all_poses_are_zero(self):
324
+ with tempfile.TemporaryDirectory() as temp:
325
+ root = Path(temp)
326
+ self._write_episode(root, [[0.0] * 7] * 10, [[0.0, 0, 0]] * 10)
327
+ with self.assertRaises(ModuleSkip):
328
+ UmiSlamActionModule().run(self._context(root))
329
+
330
+ def test_skips_when_no_parquet_present(self):
331
+ with tempfile.TemporaryDirectory() as temp:
332
+ root = Path(temp)
333
+ (root / "data").mkdir(parents=True, exist_ok=True)
334
+ with self.assertRaises(ModuleSkip):
335
+ UmiSlamActionModule().run(self._context(root))
336
+
337
+ def test_ignores_non_parquet_upstream_ref(self):
338
+ """上游 ``slam_trajectory`` 端口可能挂着一个非 parquet 产物。
339
+
340
+ worker 组装 incoming 时,若边的 sourceHandle 对不上上游任何输出,
341
+ 会把该节点的**全部输出平铺**进来 —— 视频 ref 也会落到这个键上。
342
+ 实测因此把 mp4 当 parquet 读,报 magic bytes not found。模块必须
343
+ 验证产物类型,并回退到扫描批次里的真 parquet。
344
+ """
345
+ from app.processing import ArtifactRef
346
+
347
+ with tempfile.TemporaryDirectory() as temp:
348
+ root = Path(temp)
349
+ poses = [_pose(0.01 * i + 0.004 * (i % 3), 0, 0) for i in range(12)]
350
+ self._write_episode(root, poses, [[50.0, 0, 0]] * 12)
351
+
352
+ # 上游把视频文件挂到了 slam_trajectory 这个端口上
353
+ video = root / "videos" / "observation.images.gripper_rgb" / "chunk-000"
354
+ video.mkdir(parents=True, exist_ok=True)
355
+ clip = video / "episode_000000.mp4"
356
+ clip.write_bytes(b"\x00\x00\x00\x1cftypisom" + b"\x00" * 64)
357
+
358
+ ctx = self._context(root)
359
+ ctx.incoming = {"slam_trajectory": ArtifactRef(
360
+ kind="rgb_video", path=str(clip.relative_to(root)),
361
+ source_key="gripper_rgb")}
362
+
363
+ outputs = UmiSlamActionModule().run(ctx)
364
+
365
+ self.assertIn("action", outputs, "应回退到真正的 parquet")
366
+
367
+
368
+ if __name__ == "__main__":
369
+ unittest.main()
processor/web/static/js/player.js CHANGED
@@ -1,5 +1,5 @@
1
  /* Plyr video player management */
2
- console.log('[player.js] build 202609091912-tactile-chunk-prefetch');
3
 
4
  const players = {}; // {camera_name: Plyr instance}
5
  let currentEpisodeId = null;
@@ -1219,7 +1219,15 @@ function _syncWorkspacePlayers() {
1219
  if (typeof startHeatmapSync === 'function') startHeatmapSync(active);
1220
  if (masterPlaying) {
1221
  Object.values(players).forEach(player => {
1222
- try { player.play(); } catch (e) {}
 
 
 
 
 
 
 
 
1223
  });
1224
  }
1225
  }
@@ -1781,7 +1789,13 @@ function _createTactileWebGLRenderer(canvas) {
1781
  canvas.width = width;
1782
  canvas.height = height;
1783
  }
1784
- gl.viewport(0, 0, width, height);
 
 
 
 
 
 
1785
  gl.clearColor(0.0039, 0.0039, 0.502, 1.0);
1786
  gl.clear(gl.COLOR_BUFFER_BIT);
1787
  if (!this.field) return;
@@ -1815,77 +1829,19 @@ function _createTactileWebGLRenderer(canvas) {
1815
  };
1816
  }
1817
 
1818
- // Wheel zoom / drag pan / double-click reset for one tactile matrix tile.
1819
- // One texture texel is one sensor cell, so zooming in reveals the individual
1820
- // contact cells that the whole-field view averages away (same interaction as
1821
- // the acquisition-side demo viewer).
1822
- function _tactileScreenUv(canvas, clientX, clientY) {
1823
- const rect = canvas.getBoundingClientRect();
1824
- return [
1825
- (clientX - rect.left) / Math.max(1, rect.width),
1826
- 1 - (clientY - rect.top) / Math.max(1, rect.height),
1827
- ];
1828
- }
1829
-
1830
- function _clampTactileView(renderer) {
1831
- const limit = Math.max(0, (1 - 1 / renderer.zoom) / 2);
1832
- renderer.pan[0] = Math.max(-limit, Math.min(limit, renderer.pan[0]));
1833
- renderer.pan[1] = Math.max(-limit, Math.min(limit, renderer.pan[1]));
1834
- }
1835
-
1836
- function initTactileView(entry) {
1837
- const canvas = entry?.canvas;
1838
- const renderer = entry?.renderer;
1839
- if (!canvas || !renderer) return;
1840
- const maxZoom = 12;
1841
- canvas.addEventListener('wheel', event => {
1842
- event.preventDefault();
1843
- const [u, v] = _tactileScreenUv(canvas, event.clientX, event.clientY);
1844
- const before = 1 / renderer.zoom;
1845
- renderer.zoom = Math.max(1, Math.min(maxZoom,
1846
- renderer.zoom * (event.deltaY < 0 ? 1.2 : 1 / 1.2)));
1847
- const delta = before - 1 / renderer.zoom;
1848
- renderer.pan[0] += (u - 0.5) * delta;
1849
- renderer.pan[1] += (v - 0.5) * delta;
1850
- _clampTactileView(renderer);
1851
- canvas.style.cursor = renderer.zoom > 1 ? 'grab' : 'default';
1852
- renderer.render();
1853
- }, { passive: false });
1854
- canvas.addEventListener('pointerdown', event => {
1855
- if (event.button !== 0 || renderer.zoom <= 1) return;
1856
- entry._panDrag = { x: event.clientX, y: event.clientY };
1857
- canvas.style.cursor = 'grabbing';
1858
- if (canvas.setPointerCapture) canvas.setPointerCapture(event.pointerId);
1859
- event.preventDefault();
1860
- });
1861
- canvas.addEventListener('pointermove', event => {
1862
- if (!entry._panDrag) return;
1863
- const rect = canvas.getBoundingClientRect();
1864
- renderer.pan[0] -= (event.clientX - entry._panDrag.x)
1865
- / Math.max(1, rect.width) / renderer.zoom;
1866
- renderer.pan[1] += (event.clientY - entry._panDrag.y)
1867
- / Math.max(1, rect.height) / renderer.zoom;
1868
- entry._panDrag = { x: event.clientX, y: event.clientY };
1869
- _clampTactileView(renderer);
1870
- renderer.render();
1871
- });
1872
- const endPan = event => {
1873
- if (!entry._panDrag) return;
1874
- entry._panDrag = null;
1875
- canvas.style.cursor = renderer.zoom > 1 ? 'grab' : 'default';
1876
- if (event && canvas.hasPointerCapture
1877
- && canvas.hasPointerCapture(event.pointerId)) {
1878
- canvas.releasePointerCapture(event.pointerId);
1879
- }
1880
- };
1881
- canvas.addEventListener('pointerup', endPan);
1882
- canvas.addEventListener('pointercancel', endPan);
1883
- canvas.addEventListener('dblclick', () => {
1884
- renderer.zoom = 1;
1885
- renderer.pan = [0, 0];
1886
- canvas.style.cursor = 'default';
1887
- renderer.render();
1888
- });
1889
  }
1890
 
1891
  // One chunk = 96 frames of 250x250 R8 = 6 MB per request. The old code issued
@@ -1918,7 +1874,7 @@ function _fetchTactileChunk(entry, chunkStart) {
1918
  const signal = getMediaLoadSignal();
1919
  const url = `/api/v1/video/${currentEpisodeId}/tactile-matrix-range`
1920
  + `?side=${entry.source.side}&start=${chunkStart}`
1921
- + `&limit=${TACTILE_CHUNK_FRAMES}&v=202609091912-tactile-chunk-prefetch`;
1922
  const options = { credentials: 'same-origin', cache: 'force-cache' };
1923
  if (signal) options.signal = signal;
1924
  fetch(url, options).then(response => {
@@ -2057,8 +2013,8 @@ async function mountGroupedSource(source, tile) {
2057
  curveCanvas, loadedFrame: -1, pendingFrame: null,
2058
  chunks: new Map(), chunkInflight: new Set(),
2059
  frameCount: Number(tactileMatrixData?.frame_count) || 0 };
2060
- initTactileView(entry);
2061
- canvas.title = '滚轮缩放 / 拖拽平移 / 双击复位';
2062
  currentTactileTiles.push(entry);
2063
  const frame = typeof currentFrameTarget === 'number' ? currentFrameTarget : 0;
2064
  _updateTactileForceTile(entry, frame);
@@ -2625,6 +2581,12 @@ function renderGroupedWorkspace() {
2625
  _restorePlayersToCurrentFrame();
2626
  const active = getActivePlayer();
2627
  if (!active) { hideVideoLoading(); return; }
 
 
 
 
 
 
2628
  _syncWorkspacePlayers();
2629
  refreshPlayButton(false);
2630
  try {
@@ -3324,6 +3286,10 @@ function playAll() {
3324
 
3325
 
3326
  function pauseAll() {
 
 
 
 
3327
  Object.values(players).forEach(p => {
3328
  try { p.pause(); } catch(e) {}
3329
  });
@@ -4215,10 +4181,13 @@ function _slamSmoothRows() {
4215
  // world frame: +Z is up (measured on real data - a physical turn of the
4216
  // gripper rotates about world +Z, and the gripper's own up axis sits within
4217
  // ~12 deg of it for the whole episode). The browser camera uses a right-handed
4218
- // frame with Y up, so display=[source X, source Z, -source Y] keeps the
4219
- // stored values untouched and puts world up on screen up.
 
 
 
4220
  function _slamDisplayPoint(sourcePoint) {
4221
- return [Number(sourcePoint[0]), Number(sourcePoint[2]), -Number(sourcePoint[1])];
4222
  }
4223
 
4224
  function _slamSourcePoint(row) {
@@ -4358,10 +4327,12 @@ function renderSlamTile(entry, frameOverride) {
4358
 
4359
  // Small axis marker at the fitted area: the recorded world axes, drawn in
4360
  // the display frame that _slamDisplayPoint produces (world +Z is up).
 
 
4361
  const axisOrigin = [fit.center[0], fit.gridY, fit.center[2]];
4362
  const axisTips = [
4363
  // World axes: X and Y horizontal, Z up.
4364
- [[axisOrigin[0] + 0.2, axisOrigin[1], axisOrigin[2]], 'rgb(248,90,90)', 'X'],
4365
  [[axisOrigin[0], axisOrigin[1], axisOrigin[2] - 0.2], 'rgb(80,220,130)', 'Y'],
4366
  [[axisOrigin[0], axisOrigin[1] + 0.2, axisOrigin[2]], 'rgb(90,150,255)', 'Z'],
4367
  ];
@@ -4423,10 +4394,13 @@ function renderSlamTile(entry, frameOverride) {
4423
  });
4424
 
4425
  // Simplified UMI gripper model. The recorded pose frame is the S80M
4426
- // camera frame; measured on real data the device axes sit at
4427
- // +X = camera right (~world +Y), +Y = gripper up (~world +Z, the
4428
- // camera is mounted inverted) and +Z = gripper forward (~world +X,
4429
- // the optical axis). So the body extends along local +Z, the two
 
 
 
4430
  // plates separate along local X, and each plate is 0.20m long
4431
  // (forward) by 0.05m tall (up) with a 0.12m maximum opening.
4432
  const grip = _slamGripperState(currentRow);
@@ -4449,18 +4423,18 @@ function renderSlamTile(entry, frameOverride) {
4449
  const jawFill = grip.gripped
4450
  ? 'rgba(248,90,90,0.16)' : 'rgba(88,168,232,0.12)';
4451
  const relativeCorners = [
4452
- [0, -plateHeight / 2, -plateLength / 2],
4453
- [0, -plateHeight / 2, plateLength / 2],
4454
- [0, plateHeight / 2, plateLength / 2],
4455
- [0, plateHeight / 2, -plateLength / 2],
4456
  ];
4457
 
4458
  for (const sign of [1, -1]) {
4459
  const centerX = sign * halfGap;
4460
  const corners = relativeCorners.map(corner => project(toDisplay([
4461
  centerX + corner[0],
4462
- corner[1],
4463
- baseOffset + corner[2],
4464
  ])));
4465
  if (corners.every(item => Number.isFinite(item[0]) && Number.isFinite(item[1]))) {
4466
  ctx.fillStyle = jawFill;
@@ -4474,21 +4448,21 @@ function renderSlamTile(entry, frameOverride) {
4474
  ctx.stroke();
4475
  }
4476
  line(
4477
- toDisplay([centerX, 0, baseOffset - plateLength / 2]),
4478
- toDisplay([centerX, 0, baseOffset + plateLength / 2]),
4479
  jawColor, 1);
4480
  }
4481
 
4482
  // Rear crossbar plus the two supports that tie each plate back to
4483
  // the body, so the opening stays readable when it is small.
4484
- line(toDisplay([-maxHalfGap, 0, baseOffset]),
4485
- toDisplay([maxHalfGap, 0, baseOffset]),
4486
  'rgba(148,163,184,0.8)', 1.5);
4487
  for (const sign of [1, -1]) {
4488
- for (const cornerZ of [-plateLength / 2, plateLength / 2]) {
4489
  line(
4490
- toDisplay([sign * halfGap, 0, baseOffset + cornerZ]),
4491
- toDisplay([0, 0, baseOffset + cornerZ]),
4492
  'rgba(148,163,184,0.8)', 1);
4493
  }
4494
  }
 
1
  /* Plyr video player management */
2
+ console.log('[player.js] build 20260911-tactile-episode-color-scale');
3
 
4
  const players = {}; // {camera_name: Plyr instance}
5
  let currentEpisodeId = null;
 
1219
  if (typeof startHeatmapSync === 'function') startHeatmapSync(active);
1220
  if (masterPlaying) {
1221
  Object.values(players).forEach(player => {
1222
+ // play() returns a promise: a try/catch cannot see its rejection,
1223
+ // and the load chain right after this call runs pauseAll(), which
1224
+ // aborts the pending play → uncaught AbortError in the console.
1225
+ try {
1226
+ const result = player.play();
1227
+ if (result && typeof result.catch === 'function') {
1228
+ result.catch(() => {});
1229
+ }
1230
+ } catch (e) {}
1231
  });
1232
  }
1233
  }
 
1789
  canvas.width = width;
1790
  canvas.height = height;
1791
  }
1792
+ // 传感器是 250x250 的方格阵列,必须等比显示 —— 铺满 canvas 会按
1793
+ // tile 的宽高比横向拉伸(实测宽窗约 2.9:1),格子就不再是正方形,
1794
+ // 接触位置的形状判断会失真。取短边做居中正方形 viewport,多出来的
1795
+ // 两侧留背景色(gl.clear 不受 viewport 限制,会整块清成底色)。
1796
+ const side = Math.min(width, height);
1797
+ gl.viewport(Math.round((width - side) / 2),
1798
+ Math.round((height - side) / 2), side, side);
1799
  gl.clearColor(0.0039, 0.0039, 0.502, 1.0);
1800
  gl.clear(gl.COLOR_BUFFER_BIT);
1801
  if (!this.field) return;
 
1829
  };
1830
  }
1831
 
1832
+ // The tactile force matrix tile is display-only. Zoom/pan were removed on
1833
+ // purpose: the review page is read-only evidence, and a stray wheel or drag
1834
+ // could leave the matrix zoomed into a few cells and look like a sensor
1835
+ // defect. The whole 250x250 field stays fixed, one texture texel per cell.
1836
+ // The renderer keeps supporting zoom/pan (uniforms u_zoom/u_pan) so the
1837
+ // acquisition-side demo viewer and any future opt-in can reuse it; here the
1838
+ // view is pinned to identity.
1839
+ function _resetTactileView(renderer) {
1840
+ if (!renderer) return;
1841
+ renderer.zoom = 1;
1842
+ renderer.pan = [0, 0];
1843
+ if (renderer.canvas) renderer.canvas.style.cursor = 'default';
1844
+ renderer.render();
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1845
  }
1846
 
1847
  // One chunk = 96 frames of 250x250 R8 = 6 MB per request. The old code issued
 
1874
  const signal = getMediaLoadSignal();
1875
  const url = `/api/v1/video/${currentEpisodeId}/tactile-matrix-range`
1876
  + `?side=${entry.source.side}&start=${chunkStart}`
1877
+ + `&limit=${TACTILE_CHUNK_FRAMES}&v=20260911-tactile-episode-color-scale`;
1878
  const options = { credentials: 'same-origin', cache: 'force-cache' };
1879
  if (signal) options.signal = signal;
1880
  fetch(url, options).then(response => {
 
2013
  curveCanvas, loadedFrame: -1, pendingFrame: null,
2014
  chunks: new Map(), chunkInflight: new Set(),
2015
  frameCount: Number(tactileMatrixData?.frame_count) || 0 };
2016
+ _resetTactileView(renderer);
2017
+ canvas.title = '接触力场(固定全幅,不可缩放)';
2018
  currentTactileTiles.push(entry);
2019
  const frame = typeof currentFrameTarget === 'number' ? currentFrameTarget : 0;
2020
  _updateTactileForceTile(entry, frame);
 
2581
  _restorePlayersToCurrentFrame();
2582
  const active = getActivePlayer();
2583
  if (!active) { hideVideoLoading(); return; }
2584
+ // A freshly mounted episode always starts paused. Clear the state and
2585
+ // sync before wiring the players: _syncWorkspacePlayers() re-issues
2586
+ // play() for a workspace still marked as playing, and the pauseAll()
2587
+ // further down then aborts it — a play/pause race that fires an
2588
+ // uncaught AbortError and leaves the transport button flapping.
2589
+ masterPlaying = false;
2590
  _syncWorkspacePlayers();
2591
  refreshPlayButton(false);
2592
  try {
 
3286
 
3287
 
3288
  function pauseAll() {
3289
+ // Keep the transport state consistent with reality: callers that pause
3290
+ // without flipping the flag left the button reading "Pause" while every
3291
+ // player was stopped, so the next click replayed instead of resuming.
3292
+ masterPlaying = false;
3293
  Object.values(players).forEach(p => {
3294
  try { p.pause(); } catch(e) {}
3295
  });
 
4181
  // world frame: +Z is up (measured on real data - a physical turn of the
4182
  // gripper rotates about world +Z, and the gripper's own up axis sits within
4183
  // ~12 deg of it for the whole episode). The browser camera uses a right-handed
4184
+ // frame with Y up, so the display mapping puts world up on screen up.
4185
+ //
4186
+ // 源 +X 指向设备的**左侧**(S80M R_corr 之后源系是"左、后、上"),
4187
+ // 直接透传会让整个空间视图相对真实动作左右镜像:设备往左绕圈,
4188
+ // 画面里往右走。取反 X 修正左右;Y/Z 不动,前后与上下仍与实测一致。
4189
  function _slamDisplayPoint(sourcePoint) {
4190
+ return [-Number(sourcePoint[0]), Number(sourcePoint[2]), -Number(sourcePoint[1])];
4191
  }
4192
 
4193
  function _slamSourcePoint(row) {
 
4327
 
4328
  // Small axis marker at the fitted area: the recorded world axes, drawn in
4329
  // the display frame that _slamDisplayPoint produces (world +Z is up).
4330
+ // The X tip follows the same negation as _slamDisplayPoint: display +X is
4331
+ // source -X, so the recorded +X axis points toward display -X.
4332
  const axisOrigin = [fit.center[0], fit.gridY, fit.center[2]];
4333
  const axisTips = [
4334
  // World axes: X and Y horizontal, Z up.
4335
+ [[axisOrigin[0] - 0.2, axisOrigin[1], axisOrigin[2]], 'rgb(248,90,90)', 'X'],
4336
  [[axisOrigin[0], axisOrigin[1], axisOrigin[2] - 0.2], 'rgb(80,220,130)', 'Y'],
4337
  [[axisOrigin[0], axisOrigin[1] + 0.2, axisOrigin[2]], 'rgb(90,150,255)', 'Z'],
4338
  ];
 
4394
  });
4395
 
4396
  // Simplified UMI gripper model. The recorded pose frame is the S80M
4397
+ // camera frame; measured on the uploaded UMIGripper_AI episodes (every
4398
+ // frame of every batch) the device axes sit at
4399
+ // +X = gripper right (jaw separation, horizontal),
4400
+ // +Y = gripper forward (the optical axis, horizontal) and
4401
+ // +Z = gripper up (vertical, within ~12 deg of world +Z — the world
4402
+ // frame is gravity-aligned, so this is a fixed sensor-frame fact, not
4403
+ // a per-episode pose). So the body extends along local +Y, the two
4404
  // plates separate along local X, and each plate is 0.20m long
4405
  // (forward) by 0.05m tall (up) with a 0.12m maximum opening.
4406
  const grip = _slamGripperState(currentRow);
 
4423
  const jawFill = grip.gripped
4424
  ? 'rgba(248,90,90,0.16)' : 'rgba(88,168,232,0.12)';
4425
  const relativeCorners = [
4426
+ [0, -plateLength / 2, -plateHeight / 2],
4427
+ [0, plateLength / 2, -plateHeight / 2],
4428
+ [0, plateLength / 2, plateHeight / 2],
4429
+ [0, -plateLength / 2, plateHeight / 2],
4430
  ];
4431
 
4432
  for (const sign of [1, -1]) {
4433
  const centerX = sign * halfGap;
4434
  const corners = relativeCorners.map(corner => project(toDisplay([
4435
  centerX + corner[0],
4436
+ baseOffset + corner[1],
4437
+ corner[2],
4438
  ])));
4439
  if (corners.every(item => Number.isFinite(item[0]) && Number.isFinite(item[1]))) {
4440
  ctx.fillStyle = jawFill;
 
4448
  ctx.stroke();
4449
  }
4450
  line(
4451
+ toDisplay([centerX, baseOffset - plateLength / 2, 0]),
4452
+ toDisplay([centerX, baseOffset + plateLength / 2, 0]),
4453
  jawColor, 1);
4454
  }
4455
 
4456
  // Rear crossbar plus the two supports that tie each plate back to
4457
  // the body, so the opening stays readable when it is small.
4458
+ line(toDisplay([-maxHalfGap, baseOffset, 0]),
4459
+ toDisplay([maxHalfGap, baseOffset, 0]),
4460
  'rgba(148,163,184,0.8)', 1.5);
4461
  for (const sign of [1, -1]) {
4462
+ for (const cornerY of [-plateLength / 2, plateLength / 2]) {
4463
  line(
4464
+ toDisplay([sign * halfGap, baseOffset + cornerY, 0]),
4465
+ toDisplay([0, baseOffset + cornerY, 0]),
4466
  'rgba(148,163,184,0.8)', 1);
4467
  }
4468
  }
processor/web/templates/index.html CHANGED
@@ -564,7 +564,7 @@
564
 
565
  <script src="/static/js/media-cache.js?v=202609081530"></script>
566
  <script src="/static/js/depth-renderer.js?v=202609081530"></script>
567
- <script src="/static/js/player.js?v=202609091912-tactile-chunk-prefetch"></script>
568
  <script src="/static/js/hand-overlay.js?v=202609081630"></script>
569
  <script src="/static/js/heatmap.js?v=202609081530"></script>
570
  <script src="/static/js/annotations.js?v=202609081530"></script>
 
564
 
565
  <script src="/static/js/media-cache.js?v=202609081530"></script>
566
  <script src="/static/js/depth-renderer.js?v=202609081530"></script>
567
+ <script src="/static/js/player.js?v=20260911-tactile-square-1to1"></script>
568
  <script src="/static/js/hand-overlay.js?v=202609081630"></script>
569
  <script src="/static/js/heatmap.js?v=202609081530"></script>
570
  <script src="/static/js/annotations.js?v=202609081530"></script>