Spaces:
Running
Running
stouchinglhh Claude Code commited on
Commit ·
2535d68
1
Parent(s): 383d0b9
collector: 更新采集工程至 v1.3.6(夹爪并入极简版「只录不显」,Linux 包把约 460MB 夹爪原生栈当必需资源下发并逐项自检、缺则报 [错误 B] 拒启,Windows 包有意不带、靠载荷缺失降级;三个分发壳从 lite_package/ 搬到仓库根;力矩阵编码搬到 core/gripper_codec.py 作为唯一数据契约;SLAM 轨迹点新增宿主时刻戳 observation.{prefix}slam_trajectory_ns,真机验收最近邻残差中位 6.9ms(原行号配对 83ms);另回收 native 树 339MB 重复 .so。含版本注释瘦身)
Browse files- collector/.gitignore +10 -3
- collector/README.md +109 -1
- collector/config/__init__.py +21 -1
- collector/core/egodata_writer.py +22 -3
- collector/core/gripper/bridge.py +7 -2
- collector/core/gripper/slam/protocol.py +18 -0
- collector/core/gripper_codec.py +91 -0
- collector/core/pipeline.py +37 -4
- collector/docs/data.md +37 -0
- collector/scripts/pack_lite.py +350 -38
- collector/{lite_package/start_lite.bat → start_lite.bat} +10 -1
- collector/{lite_package/start_lite.sh → start_lite.sh} +46 -1
- collector/tools/dedup_gripper_native.py +205 -0
- collector/tools/tests/lite_gripper_smoke_test.py +415 -0
- collector/tools/tests/lite_import_guard_test.py +19 -2
- collector/tools/tests/probe_gripper_rates.py +4 -1
- collector/tools/tests/smoke_gripper_p3.py +8 -2
- collector/tools/tests/test_gripper_bridge.py +22 -9
- collector/tools/tests/test_gripper_pose_axes.py +13 -1
- collector/tools/tests/test_gripper_pose_host_stamp.py +143 -0
- collector/tools/tests/test_gripper_slam_columns.py +78 -0
- collector/ui/lite_window.py +475 -19
- collector/ui/main_window.py +17 -76
- collector/使用手册.md +46 -16
- collector/{lite_package/使用说明_lite.md → 使用说明_lite.md} +89 -9
collector/.gitignore
CHANGED
|
@@ -9,6 +9,11 @@ build/
|
|
| 9 |
data/recordings/
|
| 10 |
data/*.db
|
| 11 |
*.log
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 12 |
.venv/
|
| 13 |
venv/
|
| 14 |
venv_lite/
|
|
@@ -60,7 +65,9 @@ online/
|
|
| 60 |
# ORBvoc.txt 单文件 138MB 超 GitHub 100MB 限制,且为第三方二进制,不入库)
|
| 61 |
# 触觉 Sightac SDK 例外:core/gripper/sightac_sdk/ 已 pyarmor 加密,随包入库
|
| 62 |
core/gripper/native/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 63 |
lite_package/*
|
| 64 |
-
!lite_package/start_lite.bat
|
| 65 |
-
!lite_package/start_lite.sh
|
| 66 |
-
!lite_package/使用说明_lite.md
|
|
|
|
| 9 |
data/recordings/
|
| 10 |
data/*.db
|
| 11 |
*.log
|
| 12 |
+
# 原生日志归档目录(运行期把 runtime 目录 rmtree 之前的 SLAM 原生日志留档到这里,
|
| 13 |
+
# 保留最近 20 份),连同 tools/dedup_gripper_native.py 的 JSON 报告一起**整体不入库**:
|
| 14 |
+
# 都是本机诊断产物,动辄几十 MB(实测 57MB/22 份)。`.log` 那行只挡住了日志本体,
|
| 15 |
+
# 报告是 .json 会漏网。
|
| 16 |
+
logs/
|
| 17 |
.venv/
|
| 18 |
venv/
|
| 19 |
venv_lite/
|
|
|
|
| 65 |
# ORBvoc.txt 单文件 138MB 超 GitHub 100MB 限制,且为第三方二进制,不入库)
|
| 66 |
# 触觉 Sightac SDK 例外:core/gripper/sightac_sdk/ 已 pyarmor 加密,随包入库
|
| 67 |
core/gripper/native/
|
| 68 |
+
# 极简版发布目录/本机部署目录,整体不入库。
|
| 69 |
+
# 2026-09-16 起 start_lite.bat/.sh 与 使用说明_lite.md 三个分发壳住在**仓库根**
|
| 70 |
+
# (与 main_lite.py 同级,使根目录直接可运行);lite_package/ 里那三份是
|
| 71 |
+
# scripts/pack_lite.py 收集时复制进去的产物,同样不入库——原先的 !负向规则
|
| 72 |
+
# 已随之删除,否则每次重打包都会留下未跟踪副本。
|
| 73 |
lite_package/*
|
|
|
|
|
|
|
|
|
collector/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
# collector — Multimodal Data Acquisition SDK · 多模态数据采集 SDK
|
| 2 |
|
| 3 |
-

|
| 5 |

|
| 6 |
|
|
@@ -104,6 +104,17 @@ start.bat help open the operation guide and troubleshooting doc
|
|
| 104 |
> procedure: [使用说明.md](使用说明.md) / [使用说明_EN.md](使用说明_EN.md)
|
| 105 |
> (offline wheel bundles are produced by `python scripts/pack_wheels.py`).
|
| 106 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 107 |
#### Launch the main program
|
| 108 |
|
| 109 |
```bash
|
|
@@ -143,6 +154,7 @@ venv/bin/python tools/hand_detection/demo_stereo_hands.py # stereo + MediaPipe
|
|
| 143 |
| Intel RealSense D435 / D405 | `pyrealsense2` | RGB + depth dual slots; depth recorded as 12-bit gray HEVC MP4 (log depth codes) + live JET heatmap; built-in stall/framerate watchdog with auto-reconnect; D405 has a dedicated near-range capture profile |
|
| 144 |
| S80C / S80M stereo | FaysSense VI Kit SDK (bundled in-repo, incl. FT602 bridge driver) | No SDK install needed; camera profile `STEREO_CAM_FPS` (default 50 fps) decimated to 30 fps recording via wall-clock 1/30 s buckets (burst backfill + empty-bucket watchdog, ~3% empty-bucket rate on healthy recordings); callback frame capture (same as the official GUI); carries hardware nanosecond timestamps and IMU samples; since v1.0.11 the subprocess runs the SDK depth engine → third-tile live depth heatmap + 12-bit gray depth video recording |
|
| 145 |
| BLE data gloves | `bleak` | one per hand; parquet column names bound (`right_glove` / `left_glove`) |
|
|
|
|
| 146 |
|
| 147 |
### Data Formats
|
| 148 |
|
|
@@ -319,6 +331,8 @@ file inventory, data flow):
|
|
| 319 |
- [docs/tools.md](docs/tools.md), [docs/demos.md](docs/demos.md) — 3D tools, delivery demos
|
| 320 |
- [docs/stereo_s80m.md](docs/stereo_s80m.md), [docs/hand_detection.md](docs/hand_detection.md) — S80M, hand detection
|
| 321 |
- [docs/file_format.md](docs/file_format.md) — data file interface contract (authoritative definition of the v1.1.x pooled layout)
|
|
|
|
|
|
|
| 322 |
|
| 323 |
### Privacy & Local Config
|
| 324 |
|
|
@@ -361,6 +375,55 @@ are documented in [docs/index.md](docs/index.md#开发约定).
|
|
| 361 |
|
| 362 |
### Changelog
|
| 363 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 364 |
- **v1.3.5** — the camera service heals itself, and gripper scan pairing moved
|
| 365 |
from USB root ports to NVS serials.
|
| 366 |
**Cameras:** the UVC cameras (DECXIN `1bcf:2d4f` plus two Sightac `0c45:636f`)
|
|
@@ -721,6 +784,14 @@ start.bat help 打开操作指引与异常排查文档
|
|
| 721 |
> 完整操作指引、错误码对照与内网离线交付方式见 [使用说明.md](使用说明.md) /
|
| 722 |
> [使用说明_EN.md](使用说明_EN.md)(离线安装包由 `python scripts/pack_wheels.py` 生成)。
|
| 723 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 724 |
#### 启动主程序
|
| 725 |
|
| 726 |
```bash
|
|
@@ -758,6 +829,7 @@ venv/bin/python tools/hand_detection/demo_stereo_hands.py # 双目 + MediaPipe
|
|
| 758 |
| Intel RealSense D435 / D405 | `pyrealsense2` | RGB + depth 双路槽位;深度以 12-bit 灰度 HEVC MP4 录制(对数深度码)+ 实时 JET 热力图;内置停滞/帧率看门狗自动重连;D405 有独立近距采集配置 |
|
| 759 |
| S80C / S80M 双目 | FaysSense VI Kit SDK(仓库自带,含 FT602 桥驱动) | 无需安装 SDK;相机档 `STEREO_CAM_FPS`(默认 50fps)按 wall 时钟 1/30s 桶抽帧录制 30fps(突发补录 + 空桶看门狗,健康录制空桶率 ~3%),回调取帧(官方 GUI 同款);携带硬件纳秒时间戳与 IMU 样本;v1.0.11 起子进程内置 SDK 深度引擎 → 第三格实时深度热力图 + 12-bit 灰度深度视频录制 |
|
| 760 |
| BLE 数据手套 | `bleak` | 左右手各一只,parquet 列名绑定(`right_glove` / `left_glove`) |
|
|
|
|
| 761 |
|
| 762 |
### 数据格式
|
| 763 |
|
|
@@ -922,6 +994,8 @@ venv/bin/python tools/tests/test_device_detector.py
|
|
| 922 |
- [docs/tools.md](docs/tools.md)、[docs/demos.md](docs/demos.md) — 3D 工具、交付 demo
|
| 923 |
- [docs/stereo_s80m.md](docs/stereo_s80m.md)、[docs/hand_detection.md](docs/hand_detection.md) — S80M、手部检测
|
| 924 |
- [docs/file_format.md](docs/file_format.md) — 数据文件接口契约(v1.1.x 任务池化布局权威定义)
|
|
|
|
|
|
|
| 925 |
|
| 926 |
### 隐私与本地配置
|
| 927 |
|
|
@@ -955,6 +1029,40 @@ i18n 文案经 `tr()` 翻译、PyQt5 信号参数用 `object` 封送大整数、
|
|
| 955 |
|
| 956 |
### 更新记录
|
| 957 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 958 |
- **v1.3.5** — 相机服务能自愈了;夹爪扫描配对从 USB 根端口改走 NVS 序列号。
|
| 959 |
**相机侧**:UVC 相机(DECXIN `1bcf:2d4f` 与两个 Sightac `0c45:636f`)会**静默
|
| 960 |
卡死、且不需要任何进程碰它**——2026-09-15 实证右目 Sightac 是自己夜里坏的
|
|
|
|
| 1 |
# collector — Multimodal Data Acquisition SDK · 多模态数据采集 SDK
|
| 2 |
|
| 3 |
+

|
| 4 |

|
| 5 |

|
| 6 |
|
|
|
|
| 104 |
> procedure: [使用说明.md](使用说明.md) / [使用说明_EN.md](使用说明_EN.md)
|
| 105 |
> (offline wheel bundles are produced by `python scripts/pack_wheels.py`).
|
| 106 |
|
| 107 |
+
**Lite edition** (capture + upload only — no login / playback / task page /
|
| 108 |
+
skeleton solving): double-click **`start_lite.bat`** (Windows) or run
|
| 109 |
+
`./start_lite.sh` (Linux); it uses its own `venv_lite/` (~750 MB). Same device
|
| 110 |
+
set as above, including the **UMI gripper on Linux** — there it is recorded
|
| 111 |
+
without any visualization (RGB video + force/tactile/SLAM columns only), and
|
| 112 |
+
the Linux package ships the ~460 MB gripper payload as a *required* resource
|
| 113 |
+
(`start_lite.sh` verifies 7 items up front and refuses to start with
|
| 114 |
+
`[错误 B]` if any is missing). The Windows package deliberately omits that
|
| 115 |
+
payload, so the gripper group there is always empty. See
|
| 116 |
+
[使用说明_lite.md](使用说明_lite.md).
|
| 117 |
+
|
| 118 |
#### Launch the main program
|
| 119 |
|
| 120 |
```bash
|
|
|
|
| 154 |
| Intel RealSense D435 / D405 | `pyrealsense2` | RGB + depth dual slots; depth recorded as 12-bit gray HEVC MP4 (log depth codes) + live JET heatmap; built-in stall/framerate watchdog with auto-reconnect; D405 has a dedicated near-range capture profile |
|
| 155 |
| S80C / S80M stereo | FaysSense VI Kit SDK (bundled in-repo, incl. FT602 bridge driver) | No SDK install needed; camera profile `STEREO_CAM_FPS` (default 50 fps) decimated to 30 fps recording via wall-clock 1/30 s buckets (burst backfill + empty-bucket watchdog, ~3% empty-bucket rate on healthy recordings); callback frame capture (same as the official GUI); carries hardware nanosecond timestamps and IMU samples; since v1.0.11 the subprocess runs the SDK depth engine → third-tile live depth heatmap + 12-bit gray depth video recording |
|
| 156 |
| BLE data gloves | `bleak` | one per hand; parquet column names bound (`right_glove` / `left_glove`) |
|
| 157 |
+
| UMI gripper (Fays S80M) | in-repo native stack (`core/gripper/native/`, ~460 MB, not in git) | Linux only (the native stack is ELF x86-64); up to 2 grippers (second one gets the `gripper_2_*` column prefix); records left-lens RGB video + `observation.gripper_{left,right}_force` / `_force_matrix` (10×10×3 force matrix) + `observation.slam_trajectory` + gripper state, no stereo video / IMU; new grippers auto-generate their per-serial factory calibration on open |
|
| 158 |
|
| 159 |
### Data Formats
|
| 160 |
|
|
|
|
| 331 |
- [docs/tools.md](docs/tools.md), [docs/demos.md](docs/demos.md) — 3D tools, delivery demos
|
| 332 |
- [docs/stereo_s80m.md](docs/stereo_s80m.md), [docs/hand_detection.md](docs/hand_detection.md) — S80M, hand detection
|
| 333 |
- [docs/file_format.md](docs/file_format.md) — data file interface contract (authoritative definition of the v1.1.x pooled layout)
|
| 334 |
+
- [使用说明_lite.md](使用说明_lite.md) — Lite edition guide: devices (incl. the UMI gripper), recording, upload, acceptance checklist (Chinese)
|
| 335 |
+
- [使用手册.md](使用手册.md) — full operation manual, Chinese + English
|
| 336 |
|
| 337 |
### Privacy & Local Config
|
| 338 |
|
|
|
|
| 375 |
|
| 376 |
### Changelog
|
| 377 |
|
| 378 |
+
- **v1.3.6** — the Lite edition records the UMI gripper, and every SLAM point now
|
| 379 |
+
carries the host timestamp of the frame it came from.
|
| 380 |
+
**Lite + gripper:** the Lite edition did not record the gripper at all; that
|
| 381 |
+
exclusion is now reversed. The whole gripper chain (force, 10×10×3 force
|
| 382 |
+
matrix, gripper state, SLAM trajectory) is recorded with **no gripper
|
| 383 |
+
information shown in the UI** — the gripper RGB becomes the main video source
|
| 384 |
+
directly. The payload is a hard constraint: the Linux package ships the
|
| 385 |
+
~460 MB native stack as a *required* resource and `start_lite.sh` verifies 7
|
| 386 |
+
items up front, refusing to start with `[错误 B]` if any is missing, so an
|
| 387 |
+
upgrade on an older machine is stopped before it fails halfway; the Windows
|
| 388 |
+
package deliberately omits that payload (the native stack is ELF x86-64) and
|
| 389 |
+
its gripper group stays permanently empty. The three distribution shells
|
| 390 |
+
(`start_lite.bat`, `start_lite.sh`, `使用说明_lite.md`) moved from
|
| 391 |
+
`lite_package/` to the **repository root** next to `main_lite.py`, so the root
|
| 392 |
+
is directly runnable; `lite_package/*` is now untracked entirely. Force-matrix
|
| 393 |
+
encoding moved out of `ui/main_window.py` into the new `core/gripper_codec.py`
|
| 394 |
+
(re-exported by the UI) — it is a data contract and must not exist twice, and
|
| 395 |
+
the Lite import blacklist excludes `ui/main_window`, which pulls in the whole
|
| 396 |
+
main UI. That module may depend only on numpy / `config.settings` /
|
| 397 |
+
`config.i18n`, never `core.gripper`: `core/gripper/__init__.py` imports
|
| 398 |
+
`fays_runtime` (top-level `import fcntl`) at import time, and `fcntl` does not
|
| 399 |
+
exist on Windows.
|
| 400 |
+
**SLAM host stamps:** each trajectory point's `t` is the camera *sensor* clock
|
| 401 |
+
while a video row's `hardware_ns` is the *host monotonic* clock — different
|
| 402 |
+
timebases whose offset can only be fitted (three reverse estimates spread
|
| 403 |
+
~104 ms, more than one 33.3 ms frame interval), so aligning SLAM points to
|
| 404 |
+
video frames 1:1 had been reduced to row-index pairing (median residual
|
| 405 |
+
83 ms). Native now takes `CLOCK_MONOTONIC` once at the `stereoCallback` entry
|
| 406 |
+
(before the deep copy — "the frame reached this process", not "processing
|
| 407 |
+
finished") and carries it through `RawImageFrame` → `PreparedFrame` →
|
| 408 |
+
`OutputFrame` (the prepared-frame slot is latest-wins, so timestamp and stamp
|
| 409 |
+
must travel together or they get paired with the wrong frame) out to the
|
| 410 |
+
`Host:(<ns>)` field on stdout. `protocol.py` parses it as an **optional**
|
| 411 |
+
group — deployed older binaries do not print it and still parse, with
|
| 412 |
+
`host_mono_ns = None` — the bridge adds a 5th **object** signal argument
|
| 413 |
+
(`pyqtSignal(int)` would silently truncate nanoseconds to qint32), and it
|
| 414 |
+
lands as `observation.{prefix}slam_trajectory_ns` (`list<int64>`, declared
|
| 415 |
+
`flat_parallel_to_slam_trajectory` / `host_monotonic_ns`), accumulated in
|
| 416 |
+
lockstep with the point list: one value per point, 0 = unknown, never skipped,
|
| 417 |
+
and no column at all when a whole segment is unstamped. Verified on real
|
| 418 |
+
hardware with episode-085/086 (the first two recorded after the deploy; the
|
| 419 |
+
earlier 128 carry no such column): points = stamps (353/353 and 344/344), zero
|
| 420 |
+
length mismatches, zero all-zero rows, strictly increasing, stamped range
|
| 421 |
+
inside the segment's `hardware_ns` range (same timebase) — nearest-neighbour
|
| 422 |
+
residual median 6.9 ms with **94.6 % / 97.4 % inside half a frame interval**,
|
| 423 |
+
against 83 ms for row-index pairing. Also adds
|
| 424 |
+
`tools/dedup_gripper_native.py`, which recovers 339 MB of `.so` files that a
|
| 425 |
+
ZIP / `cp -rL` transfer had dereferenced from symlinks into full copies (the
|
| 426 |
+
worst case being one libopenblas stored 7 times in orb48_env).
|
| 427 |
- **v1.3.5** — the camera service heals itself, and gripper scan pairing moved
|
| 428 |
from USB root ports to NVS serials.
|
| 429 |
**Cameras:** the UVC cameras (DECXIN `1bcf:2d4f` plus two Sightac `0c45:636f`)
|
|
|
|
| 784 |
> 完整操作指引、错误码对照与内网离线交付方式见 [使用说明.md](使用说明.md) /
|
| 785 |
> [使用说明_EN.md](使用说明_EN.md)(离线安装包由 `python scripts/pack_wheels.py` 生成)。
|
| 786 |
|
| 787 |
+
**极简版**(只做连接设备 → 采集 → 上传,无登录/回放/任务页/骨架解算):
|
| 788 |
+
Windows 双击 **`start_lite.bat`**、Linux 跑 `./start_lite.sh`,用独立环境
|
| 789 |
+
`venv_lite/`(约 750MB)。设备范围同上,**含 UMI 夹爪(仅 Linux)**——夹爪
|
| 790 |
+
**只录不显**(只落 RGB 视频 + 力/触觉/力矩阵/SLAM 轨迹列,界面上不显示),
|
| 791 |
+
且 Linux 包把约 460MB 的夹爪原生载荷当**必需资源**下发(`start_lite.sh`
|
| 792 |
+
启动前逐项校验 7 项,缺了直接报 `[错误 B]` 拒启);Windows 包有意不带该载荷,
|
| 793 |
+
那边的「UMI 夹爪」组框还在、里面永远是空的。详见 [使用说明_lite.md](使用说明_lite.md)。
|
| 794 |
+
|
| 795 |
#### 启动主程序
|
| 796 |
|
| 797 |
```bash
|
|
|
|
| 829 |
| Intel RealSense D435 / D405 | `pyrealsense2` | RGB + depth 双路槽位;深度以 12-bit 灰度 HEVC MP4 录制(对数深度码)+ 实时 JET 热力图;内置停滞/帧率看门狗自动重连;D405 有独立近距采集配置 |
|
| 830 |
| S80C / S80M 双目 | FaysSense VI Kit SDK(仓库自带,含 FT602 桥驱动) | 无需安装 SDK;相机档 `STEREO_CAM_FPS`(默认 50fps)按 wall 时钟 1/30s 桶抽帧录制 30fps(突发补录 + 空桶看门狗,健康录制空桶率 ~3%),回调取帧(官方 GUI 同款);携带硬件纳秒时间戳与 IMU 样本;v1.0.11 起子进程内置 SDK 深度引擎 → 第三格实时深度热力图 + 12-bit 灰度深度视频录制 |
|
| 831 |
| BLE 数据手套 | `bleak` | 左右手各一只,parquet 列名绑定(`right_glove` / `left_glove`) |
|
| 832 |
+
| UMI 夹爪(Fays S80M) | 仓库自带原生栈(`core/gripper/native/`,约 460MB,不入库) | 仅 Linux(原生栈是 ELF x86-64);最多 2 台(第二台的数据列自动加 `gripper_2_*` 前缀);录左目 RGB 视频 + `observation.gripper_{left,right}_force` / `_force_matrix`(10×10×3 力矩阵)+ `observation.slam_trajectory` + 夹爪状态,不存双目视频/IMU;新夹爪开启时自动生成逐台出厂标定 |
|
| 833 |
|
| 834 |
### 数据格式
|
| 835 |
|
|
|
|
| 994 |
- [docs/tools.md](docs/tools.md)、[docs/demos.md](docs/demos.md) — 3D 工具、交付 demo
|
| 995 |
- [docs/stereo_s80m.md](docs/stereo_s80m.md)、[docs/hand_detection.md](docs/hand_detection.md) — S80M、手部检测
|
| 996 |
- [docs/file_format.md](docs/file_format.md) — 数据文件接口契约(v1.1.x 任务池化布局权威定义)
|
| 997 |
+
- [使用说明_lite.md](使用说明_lite.md) — 极简版使用说明:设备(含 UMI 夹爪)、录制、上传、验收清单
|
| 998 |
+
- [使用手册.md](使用手册.md) — 完整操作手册(中文 + English)
|
| 999 |
|
| 1000 |
### 隐私与本地配置
|
| 1001 |
|
|
|
|
| 1029 |
|
| 1030 |
### 更新记录
|
| 1031 |
|
| 1032 |
+
- **v1.3.6** — 极简版能录 UMI 夹爪了;每个 SLAM 点带上了它那张取样帧的宿主时刻。
|
| 1033 |
+
**极简版 + 夹爪**:极简版此前完全不录夹爪,本次推翻该排除。夹爪整条链(力 /
|
| 1034 |
+
10×10×3 力矩阵 / 夹爪状态 / SLAM 轨迹)全部落盘,但界面**不显示**任何夹爪
|
| 1035 |
+
信息——夹爪 RGB 直接当主视频源用。载荷是硬约束:Linux 包把约 460MB 的原生
|
| 1036 |
+
栈当**必需资源**下发,`start_lite.sh` 启动前逐项校验 7 项,缺任一项报
|
| 1037 |
+
`[错误 B]` 拒启,老机器升级会在跑炸之前就被拦住;Windows 包**有意不带**该
|
| 1038 |
+
载荷(原生栈是 ELF x86-64),那边的夹爪组框永远是空的。三个分发壳
|
| 1039 |
+
(`start_lite.bat`/`start_lite.sh`/`使用说明_lite.md`)从 `lite_package/`
|
| 1040 |
+
搬到**仓库根**、与 `main_lite.py` 同级,根目录直接可运行;`lite_package/*`
|
| 1041 |
+
随之整体不入库。力矩阵编码从 `ui/main_window.py` 搬到新模块
|
| 1042 |
+
`core/gripper_codec.py`(UI 侧 re-export)——它是数据契约,不能在两份界面
|
| 1043 |
+
代码里各存一份,而极简版的导入黑名单不含 `ui/main_window`(它拖进主程序
|
| 1044 |
+
全部 UI)。该模块只许依赖 numpy / `config.settings` / `config.i18n`,绝不
|
| 1045 |
+
可 import `core.gripper`:`core/gripper/__init__.py` 导入期就拉
|
| 1046 |
+
`fays_runtime`(顶层 `import fcntl`),而 fcntl 在 Windows 上不存在。
|
| 1047 |
+
**SLAM 宿主戳**:轨迹每点的 `t` 是**相机传感器钟**,视频行的 `hardware_ns`
|
| 1048 |
+
是**宿主单调钟**,两者不同源、只差一个只能拟合的偏移(三个反推估计散布
|
| 1049 |
+
~104ms,已超一个 33.3ms 帧间隔),于是把它们 1:1 对齐此前只能退化成按行号
|
| 1050 |
+
配(中位残差 83ms)。现在 native 在 `stereoCallback` 入口取一次
|
| 1051 |
+
`CLOCK_MONOTONIC`(在深拷贝之前——代表「帧到达本进程」而非「处理完」),
|
| 1052 |
+
随帧走完 `RawImageFrame` → `PreparedFrame` → `OutputFrame`(预处理槽是
|
| 1053 |
+
「最新覆盖」,所以时间戳与宿主戳必须同进同出,否则会配错帧),与位姿一起
|
| 1054 |
+
打在 stdout 的 `Host:(<ns>)` 上。`protocol.py` 按**可选**组解析——现场已
|
| 1055 |
+
部署的旧二进制不打印这段、照旧可读,`host_mono_ns = None`;桥接加第 5 个
|
| 1056 |
+
**object** 信号参(`pyqtSignal(int)` 会把纳秒按 qint32 静默截断);最终落成
|
| 1057 |
+
`observation.{prefix}slam_trajectory_ns`(`list<int64>`,声明
|
| 1058 |
+
`flat_parallel_to_slam_trajectory` / `host_monotonic_ns`),**与点列表锁步**
|
| 1059 |
+
累加:每点恒定一个值、无戳补 0、绝不跳过,整段全无戳则不建列。真机验收
|
| 1060 |
+
(部署后录的 episode-085/086,正是头两段带该列的;此前 128 段都没有):
|
| 1061 |
+
点数=戳数(353/353、344/344)、零长度不等、零全 0 行、严格递增、戳区间落在
|
| 1062 |
+
该段 `hardware_ns` 区间内(同一时基)——最近邻残差中位 6.9ms、**≤半帧占比
|
| 1063 |
+
94.6% / 97.4%**,而行号配对是中位 83ms。另新增
|
| 1064 |
+
`tools/dedup_gripper_native.py`,回收 339MB 被 ZIP / `cp -rL` 从软链解引用
|
| 1065 |
+
成实体副本的 `.so`(最夸张的是 orb48_env 里同一个 libopenblas 存了 7 份)。
|
| 1066 |
- **v1.3.5** — 相机服务能自愈了;夹爪扫描配对从 USB 根端口改走 NVS 序列号。
|
| 1067 |
**相机侧**:UVC 相机(DECXIN `1bcf:2d4f` 与两个 Sightac `0c45:636f`)会**静默
|
| 1068 |
卡死、且不需要任何进程碰它**——2026-09-15 实证右目 Sightac 是自己夜里坏的
|
collector/config/__init__.py
CHANGED
|
@@ -3,4 +3,24 @@
|
|
| 3 |
版本号集中定义于此:settings.APP_VERSION 及其他模块均从这里取值。
|
| 4 |
"""
|
| 5 |
|
| 6 |
-
__version__ = "1.3.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
版本号集中定义于此:settings.APP_VERSION 及其他模块均从这里取值。
|
| 4 |
"""
|
| 5 |
|
| 6 |
+
__version__ = "1.3.6"
|
| 7 |
+
|
| 8 |
+
# v1.3.6 —— 夹爪并入极简版(只录不显)+ SLAM 轨迹点带上取样帧的宿主时刻戳。
|
| 9 |
+
# 1) 极简版新增夹爪:力 / 10×10×3 力矩阵 / 夹爪状态 / SLAM 轨迹照常落盘,界面
|
| 10 |
+
# 不显示任何夹爪信息,夹爪 RGB 直接当主视频源。载荷是硬约束——Linux 包把
|
| 11 |
+
# ~460MB 原生栈当必需资源下发,start_lite.sh 逐项校验、缺则报 [错误 B] 拒启
|
| 12 |
+
# (老机器升级会被拦住而不是跑一半才炸);Windows 包有意不带、靠载荷缺失降级。
|
| 13 |
+
# 力矩阵编码搬到 core/gripper_codec.py(数据契约只此一份,UI 侧 re-export)。
|
| 14 |
+
# 2) SLAM 点新增 observation.{prefix}slam_trajectory_ns:native 在 stereoCallback
|
| 15 |
+
# 入口取一次 CLOCK_MONOTONIC,随帧走到 stdout 的 Host:(<ns>),与点列表锁步落盘。
|
| 16 |
+
# 真机验收(episode-085/086):点数=戳数、零全 0 行、最近邻残差中位 6.9ms
|
| 17 |
+
# (按行号配对为 83ms)。旧 native 二进制不打印该字段,照旧可读。
|
| 18 |
+
# 另回收 native 树 339MB 被解引用成实体副本的 .so(tools/dedup_gripper_native.py)。
|
| 19 |
+
#
|
| 20 |
+
# 历史版本一行一条,完整说明见 README 的「更新记录」:
|
| 21 |
+
# v1.3.5 相机服务静默卡死自愈;夹爪扫描配对改走 ESP32 NVS 序列号
|
| 22 |
+
# v1.3.4 新夹爪接入即自动生成运行标定并连接,不再需要人工两步
|
| 23 |
+
# v1.3.3 力矩阵 / 触觉力与 RGB 视频的时间戳对齐(补 capture_ns 稀疏列)
|
| 24 |
+
# v1.3.2 SLAM 陈旧帧入库前拦截;轨迹显示去掉主线程深拷贝
|
| 25 |
+
# v1.3.1 力矩阵落盘规格改 5 档可选,倍率记在每段一份的 meta 行
|
| 26 |
+
# v1.3.0 UMI 夹爪全链接入(Fays S80M 双目 SLAM 位姿/轨迹 + 触觉力 + RGB)
|
collector/core/egodata_writer.py
CHANGED
|
@@ -730,7 +730,13 @@ class EgoDataWriter(QObject):
|
|
| 730 |
for key, value in (gripper or {}).items():
|
| 731 |
if value is None:
|
| 732 |
continue
|
| 733 |
-
if key.endswith("
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 734 |
# 采集时刻列(int64 宿主单调钟纳秒,与 hardware_ns 同时基)。
|
| 735 |
# 后缀判定必须排在 _force_matrix/_force 之前:`_force_matrix_ns`
|
| 736 |
# 不以 `_force_matrix` 结尾只是巧合,靠顺序防止以后改后缀踩雷。
|
|
@@ -919,7 +925,12 @@ class EgoDataWriter(QObject):
|
|
| 919 |
# 后者类型按本 episode 实际规格:int16 行差分 / float32 原值)
|
| 920 |
for key in sorted(self._present_gripper):
|
| 921 |
name = f"observation.{key}"
|
| 922 |
-
if key.endswith("
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 923 |
# 采集时刻(缺失帧填 0 = 未知,与 hardware_ns 的 0 约定一致)
|
| 924 |
cols[name] = pa.array(
|
| 925 |
[int(r.get(name, 0)) for r in rows], pa.int64())
|
|
@@ -1055,7 +1066,15 @@ class EgoDataWriter(QObject):
|
|
| 1055 |
# 这里不重写一遍以免两处漂移)
|
| 1056 |
matrix_features = self._force_matrix_features()
|
| 1057 |
for key in sorted(self._present_gripper):
|
| 1058 |
-
if key.endswith("
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1059 |
# 采集时刻:宿主单调钟纳秒,与 hardware_ns 同一时基。
|
| 1060 |
# 下游按此把力样本重采样到 RGB 帧时刻(见 docs/data.md)。
|
| 1061 |
features[f"observation.{key}"] = {
|
|
|
|
| 730 |
for key, value in (gripper or {}).items():
|
| 731 |
if value is None:
|
| 732 |
continue
|
| 733 |
+
if key.endswith("slam_trajectory_ns"):
|
| 734 |
+
# 与 slam_trajectory 逐点同序并行的宿主时刻(变长 int64
|
| 735 |
+
# 纳秒)。**必须排在通用 `_ns` 之前**:后者按"每行一个
|
| 736 |
+
# 标量 int64"落盘,而一行内可以有多达 10 个轨迹点,
|
| 737 |
+
# 落成标量就只剩一个时刻、与点列表再也对不上。
|
| 738 |
+
row[f"observation.{key}"] = [int(v) for v in value]
|
| 739 |
+
elif key.endswith("_ns"):
|
| 740 |
# 采集时刻列(int64 宿主单调钟纳秒,与 hardware_ns 同时基)。
|
| 741 |
# 后缀判定必须排在 _force_matrix/_force 之前:`_force_matrix_ns`
|
| 742 |
# 不以 `_force_matrix` 结尾只是巧合,靠顺序防止以后改后缀踩雷。
|
|
|
|
| 925 |
# 后者类型按本 episode 实际规格:int16 行差分 / float32 原值)
|
| 926 |
for key in sorted(self._present_gripper):
|
| 927 |
name = f"observation.{key}"
|
| 928 |
+
if key.endswith("slam_trajectory_ns"):
|
| 929 |
+
# 逐点宿主时刻,与 slam_trajectory **同序等长**;某帧无
|
| 930 |
+
# 轨迹点则空列表(与 slam_trajectory 的空列表同步)。
|
| 931 |
+
cols[name] = pa.array(
|
| 932 |
+
[r.get(name, []) for r in rows], pa.list_(pa.int64()))
|
| 933 |
+
elif key.endswith("_ns"):
|
| 934 |
# 采集时刻(缺失帧填 0 = 未知,与 hardware_ns 的 0 约定一致)
|
| 935 |
cols[name] = pa.array(
|
| 936 |
[int(r.get(name, 0)) for r in rows], pa.int64())
|
|
|
|
| 1066 |
# 这里不重写一遍以免两处漂移)
|
| 1067 |
matrix_features = self._force_matrix_features()
|
| 1068 |
for key in sorted(self._present_gripper):
|
| 1069 |
+
if key.endswith("slam_trajectory_ns"):
|
| 1070 |
+
# 逐点宿主时刻(int64 ns),与 slam_trajectory 同序等长:
|
| 1071 |
+
# shape [1] 指"每个轨迹点 1 个值",对齐前者的 [8]。
|
| 1072 |
+
# 下游按它与 hardware_ns 直接比较即可配 slam 点↔视频帧。
|
| 1073 |
+
features[f"observation.{key}"] = {
|
| 1074 |
+
"dtype": "int64", "shape": [1],
|
| 1075 |
+
"encoding": "flat_parallel_to_slam_trajectory",
|
| 1076 |
+
"clock": "host_monotonic_ns"}
|
| 1077 |
+
elif key.endswith("_ns"):
|
| 1078 |
# 采集时刻:宿主单调钟纳秒,与 hardware_ns 同一时基。
|
| 1079 |
# 下游按此把力样本重采样到 RGB 帧时刻(见 docs/data.md)。
|
| 1080 |
features[f"observation.{key}"] = {
|
collector/core/gripper/bridge.py
CHANGED
|
@@ -183,7 +183,7 @@ class GripperBridge(QObject):
|
|
| 183 |
# 符号,下游拿它做不了任何对齐(且解回卷在 ±2.147s 歧义边界上
|
| 184 |
# 猜错会凭空造出上百帧的假漂移)。与 main_window 的
|
| 185 |
# stereo_frame_ready 用 object 同理。
|
| 186 |
-
pose_ready = pyqtSignal(object, object, object, object)
|
| 187 |
# pos(x,y,z), quat(qx,qy,qz,qw), trajectory(位置点不可变元组,新点追加才重建),
|
| 188 |
# timestamp(SLAM 秒,与轨迹 txt 第一列同源——轨迹并入 episode parquet 后
|
| 189 |
# 每点 8 值 [t,x,y,z,qx,qy,qz,qw] 需要它)
|
|
@@ -625,7 +625,12 @@ class GripperBridge(QObject):
|
|
| 625 |
self._trajectory_points[-TRAJECTORY_POINT_CAP:])
|
| 626 |
self._trajectory_cache = tuple(self._trajectory_points)
|
| 627 |
self._post("pose", tuple(pose.position), tuple(pose.rotation),
|
| 628 |
-
self._trajectory_cache, pose.timestamp
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 629 |
|
| 630 |
def _on_stereo_packet(
|
| 631 |
self, payload, sensor_ts_ns, host_mono_ns, sequence,
|
|
|
|
| 183 |
# 符号,下游拿它做不了任何对齐(且解回卷在 ±2.147s 歧义边界上
|
| 184 |
# 猜错会凭空造出上百帧的假漂移)。与 main_window 的
|
| 185 |
# stereo_frame_ready 用 object 同理。
|
| 186 |
+
pose_ready = pyqtSignal(object, object, object, object, object)
|
| 187 |
# pos(x,y,z), quat(qx,qy,qz,qw), trajectory(位置点不可变元组,新点追加才重建),
|
| 188 |
# timestamp(SLAM 秒,与轨迹 txt 第一列同源——轨迹并入 episode parquet 后
|
| 189 |
# 每点 8 值 [t,x,y,z,qx,qy,qz,qw] 需要它)
|
|
|
|
| 625 |
self._trajectory_points[-TRAJECTORY_POINT_CAP:])
|
| 626 |
self._trajectory_cache = tuple(self._trajectory_points)
|
| 627 |
self._post("pose", tuple(pose.position), tuple(pose.rotation),
|
| 628 |
+
self._trajectory_cache, pose.timestamp,
|
| 629 |
+
# 取样帧的宿主单调钟纳秒(与 hardware_ns 同时基);旧
|
| 630 |
+
# native 二进制不打印 Host 字段 → None,下游按未知处理。
|
| 631 |
+
# getattr 而非直接取属性:测试夹具/外部实现可能只有
|
| 632 |
+
# pos/quat/ts 三个字段,缺戳不应让整条位姿断流。
|
| 633 |
+
getattr(pose, "host_mono_ns", None))
|
| 634 |
|
| 635 |
def _on_stereo_packet(
|
| 636 |
self, payload, sensor_ts_ns, host_mono_ns, sequence,
|
collector/core/gripper/slam/protocol.py
CHANGED
|
@@ -78,6 +78,9 @@ POSE_MAX_ANGULAR_SPEED_RAD_S = math.radians(1080.0)
|
|
| 78 |
_POSE_RE = re.compile(
|
| 79 |
r"\[([\d.]+)\]\s*XYZ:\(([-\d.e+]+),([-\d.e+]+),([-\d.e+]+)\)\s*"
|
| 80 |
r"Quat:\(w=([-\d.e+]+),x=([-\d.e+]+),y=([-\d.e+]+),z=([-\d.e+]+)\)"
|
|
|
|
|
|
|
|
|
|
| 81 |
)
|
| 82 |
_FPS_RE = re.compile(
|
| 83 |
r"^\[FPS_DATA\]\s+image=([\d.]+)\s+imu=([\d.]+)\s+"
|
|
@@ -115,6 +118,13 @@ class PoseSample:
|
|
| 115 |
sequence: int = 0
|
| 116 |
arrival_monotonic: object = None
|
| 117 |
arrival_wall_time: object = None
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 118 |
|
| 119 |
@property
|
| 120 |
def valid(self):
|
|
@@ -125,6 +135,7 @@ class PoseSample:
|
|
| 125 |
arrival_values = (
|
| 126 |
self.arrival_monotonic,
|
| 127 |
self.arrival_wall_time,
|
|
|
|
| 128 |
)
|
| 129 |
return (
|
| 130 |
math.isfinite(self.timestamp)
|
|
@@ -354,12 +365,16 @@ def _parse_pose(line):
|
|
| 354 |
qw, qx, qy, qz = (
|
| 355 |
float(match.group(index)) for index in range(5, 9)
|
| 356 |
)
|
|
|
|
|
|
|
|
|
|
| 357 |
except (TypeError, ValueError, OverflowError):
|
| 358 |
return None
|
| 359 |
return PoseSample(
|
| 360 |
position=(x, y, z),
|
| 361 |
rotation=(qx, qy, qz, qw),
|
| 362 |
timestamp=timestamp,
|
|
|
|
| 363 |
)
|
| 364 |
|
| 365 |
|
|
@@ -390,6 +405,9 @@ def rotate_pose_z90(pose):
|
|
| 390 |
rotation=(c * (qx - qy), c * (qy + qx), c * (qz + qw),
|
| 391 |
c * (qw - qz)),
|
| 392 |
timestamp=pose.timestamp,
|
|
|
|
|
|
|
|
|
|
| 393 |
)
|
| 394 |
|
| 395 |
|
|
|
|
| 78 |
_POSE_RE = re.compile(
|
| 79 |
r"\[([\d.]+)\]\s*XYZ:\(([-\d.e+]+),([-\d.e+]+),([-\d.e+]+)\)\s*"
|
| 80 |
r"Quat:\(w=([-\d.e+]+),x=([-\d.e+]+),y=([-\d.e+]+),z=([-\d.e+]+)\)"
|
| 81 |
+
# 取样帧的宿主单调钟纳秒(native 打印的 Host:(<ns)>)。**可选**:
|
| 82 |
+
# 现场已部署的旧 native 二进制不打印这段,缺省即 None,老数据照旧可读。
|
| 83 |
+
r"(?:\s+Host:\((\d+)\))?"
|
| 84 |
)
|
| 85 |
_FPS_RE = re.compile(
|
| 86 |
r"^\[FPS_DATA\]\s+image=([\d.]+)\s+imu=([\d.]+)\s+"
|
|
|
|
| 118 |
sequence: int = 0
|
| 119 |
arrival_monotonic: object = None
|
| 120 |
arrival_wall_time: object = None
|
| 121 |
+
# 该取样帧进入 native 进程时的宿主单调钟纳秒(CLOCK_MONOTONIC),与
|
| 122 |
+
# 主程序侧 `hardware_ns` / `time.monotonic_ns()` **同一时基**。
|
| 123 |
+
# `timestamp` 是相机传感器钟,与宿主钟不同源:两者只差一个近似常量
|
| 124 |
+
# 但带抖动的偏移,只能拟合不能换算(实测 offset 反推值散布 ~104ms,
|
| 125 |
+
# 已超一个帧间隔)。有本字段才能把 slam 点与视频帧按时间直接对齐。
|
| 126 |
+
# None = 无戳(旧 native 二进制),下游按"未知"处理、退回行号配对。
|
| 127 |
+
host_mono_ns: object = None
|
| 128 |
|
| 129 |
@property
|
| 130 |
def valid(self):
|
|
|
|
| 135 |
arrival_values = (
|
| 136 |
self.arrival_monotonic,
|
| 137 |
self.arrival_wall_time,
|
| 138 |
+
self.host_mono_ns,
|
| 139 |
)
|
| 140 |
return (
|
| 141 |
math.isfinite(self.timestamp)
|
|
|
|
| 365 |
qw, qx, qy, qz = (
|
| 366 |
float(match.group(index)) for index in range(5, 9)
|
| 367 |
)
|
| 368 |
+
# 组 9 = Host:(<ns>),旧二进制的位姿行没这段 → None
|
| 369 |
+
host_group = match.group(9)
|
| 370 |
+
host_mono_ns = int(host_group) if host_group is not None else None
|
| 371 |
except (TypeError, ValueError, OverflowError):
|
| 372 |
return None
|
| 373 |
return PoseSample(
|
| 374 |
position=(x, y, z),
|
| 375 |
rotation=(qx, qy, qz, qw),
|
| 376 |
timestamp=timestamp,
|
| 377 |
+
host_mono_ns=host_mono_ns,
|
| 378 |
)
|
| 379 |
|
| 380 |
|
|
|
|
| 405 |
rotation=(c * (qx - qy), c * (qy + qx), c * (qz + qw),
|
| 406 |
c * (qw - qz)),
|
| 407 |
timestamp=pose.timestamp,
|
| 408 |
+
# 纯世界重标,不动时基:宿主戳原样透传(丢了它位姿就退回
|
| 409 |
+
# "只能按行号/相机钟对齐"的状态,落盘侧拿不到真实取样时刻)
|
| 410 |
+
host_mono_ns=pose.host_mono_ns,
|
| 411 |
)
|
| 412 |
|
| 413 |
|
collector/core/gripper_codec.py
ADDED
|
@@ -0,0 +1,91 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""夹爪触觉力矩阵编码 —— 落盘契约的中立家。
|
| 2 |
+
|
| 3 |
+
从 ui/main_window.py 搬来(2026-09-16),原因:极简版(lite)也要录夹爪,
|
| 4 |
+
但 lite 的导入黑名单不含 ui/main_window(它拖进主程序全部 UI 与重依赖),
|
| 5 |
+
而力矩阵编码是**数据契约**,绝不能在两份界面代码里各存一份。
|
| 6 |
+
|
| 7 |
+
命名镜像既有的 core/depth_codec.py(同为「算法在家、UI 只管显示」的分工)。
|
| 8 |
+
|
| 9 |
+
★ 本模块**只能**依赖 numpy / config.settings / config.i18n。
|
| 10 |
+
绝不可 import core.gripper —— core/gripper/__init__.py 在导入期就会拉
|
| 11 |
+
fays_runtime(顶层 `import fcntl`),而 fcntl 在 Windows 上不存在,
|
| 12 |
+
一旦引入,主程序 `import ui.main_window` 会在 Windows 上直接崩
|
| 13 |
+
(ui/main_window.py 从这里 re-export 这两个函数)。
|
| 14 |
+
"""
|
| 15 |
+
|
| 16 |
+
from __future__ import annotations
|
| 17 |
+
|
| 18 |
+
import numpy as np
|
| 19 |
+
|
| 20 |
+
from config import settings
|
| 21 |
+
from config.i18n import tr
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
def encode_gripper_force_matrix(m: np.ndarray, spec: str = "int16") -> list:
|
| 25 |
+
"""触觉力矩阵编码(P4 泵线程,返回扁平数值列表)。
|
| 26 |
+
|
| 27 |
+
spec 取 settings.GRIPPER_FORCE_MATRIX_SPECS 之一。
|
| 28 |
+
|
| 29 |
+
**注意**:省略 spec 时走的是函数签名默认 "int16"(兼容历史调用的冻结
|
| 30 |
+
契约),**不跟随工具栏的默认档**;生产路径(P4 泵)一律显式传入录制
|
| 31 |
+
开始时锁定的 spec,绝不省略。
|
| 32 |
+
|
| 33 |
+
"int16"(既有契约):不缩放,与
|
| 34 |
+
online/recording/lerobot_v3.quantize_force_matrix 一致的有符号补码行
|
| 35 |
+
差分(mod 2^16 可逆),扩展到 (250,250,3) 三力平面:每行 250×3 个
|
| 36 |
+
元素横向差分;超出 ±32767 mN 饱和截断。**逐点 astype(int16) 是向零
|
| 37 |
+
截断,小数部分全部丢失**——实测真实录制里逐点值只剩 -3…8 的整数电平,
|
| 38 |
+
亚毫牛梯度(接触形状)不可恢复。保留向零截断是为了与历史录制逐位一致。
|
| 39 |
+
|
| 40 |
+
"int16xN"(N=10/100/1000,定标量化):先 ×N 再四舍五入到整数存 int16,
|
| 41 |
+
解码端除以 N 还原,分辨率 1/N mN。饱和上限相应变为 ±32767/N mN
|
| 42 |
+
(×1000 即 ±32.767 mN)。定标档用 rint 而非截断:没有历史包袱,且
|
| 43 |
+
四舍五入把最大误差减半、不留系统性偏零。行差分与 int16 档完全相同,
|
| 44 |
+
所以仍然压得很小(×10 实测 0.122 字节/点,float32 的 1/6.5)。
|
| 45 |
+
|
| 46 |
+
"float32"(原值直存):不做任何量化,把 (250,250,3) 按行展平成 250×750
|
| 47 |
+
的 float32 列表。无截断、无饱和,误差为 float32 本身的 ~1e-7 相对精度。
|
| 48 |
+
体积换精度:实测 0.797 字节/点(本段数据约为 int16 档的 119 倍)。
|
| 49 |
+
|
| 50 |
+
反解:整数档 reshape(-1,750) → cumsum(axis=1) → reshape(250,250,3)
|
| 51 |
+
(cumsum 走 int32 再截断回 int16 恢复 mod 2^16 回绕),
|
| 52 |
+
再按倍率 ÷N;float32 档 reshape(-1,750) → (250,250,3)。
|
| 53 |
+
|
| 54 |
+
★ 返回的 Python 元素类型只区分「家族」:int = int16 系(**倍率无法从
|
| 55 |
+
元素类型看出**,×10 和 ×1000 都是 int),float = float32。
|
| 56 |
+
倍率必须由 info.json features.scale 携带,writer 与 demo 都按
|
| 57 |
+
scale + 列类型共同判别,不要只看元素类型。
|
| 58 |
+
"""
|
| 59 |
+
if spec not in settings.GRIPPER_FORCE_MATRIX_SPECS:
|
| 60 |
+
raise ValueError(f"未知力矩阵规格 {spec!r}")
|
| 61 |
+
arr = np.asarray(m, dtype=np.float32)
|
| 62 |
+
if spec == "float32":
|
| 63 |
+
# 与整数档同为「扁平 250×750 列表」,只有元素类型不同:
|
| 64 |
+
# 读取端统一 reshape(-1,750) 后再按 dtype 决定是否 cumsum
|
| 65 |
+
return arr.reshape(-1).tolist()
|
| 66 |
+
scale = settings.GRIPPER_FORCE_MATRIX_SCALES[spec]
|
| 67 |
+
rows = int(arr.shape[0])
|
| 68 |
+
flat = arr.reshape(rows, -1)
|
| 69 |
+
if scale == 1:
|
| 70 |
+
q = np.clip(flat, -32767.0, 32767.0).astype(np.int16)
|
| 71 |
+
else:
|
| 72 |
+
# rint 后已是整数值,astype 的向零截断不再丢任何东西
|
| 73 |
+
q = np.clip(np.rint(flat * np.float32(scale)),
|
| 74 |
+
-32767.0, 32767.0).astype(np.int16)
|
| 75 |
+
d = np.empty_like(q)
|
| 76 |
+
d[:, 0] = q[:, 0]
|
| 77 |
+
d[:, 1:] = q[:, 1:] - q[:, :-1] # int16 mod 2^16 回绕
|
| 78 |
+
return np.frombuffer(d.tobytes(), dtype=np.int16).tolist()
|
| 79 |
+
|
| 80 |
+
|
| 81 |
+
def describe_gripper_matrix_spec(spec: str) -> str:
|
| 82 |
+
"""规格 → 人话(工具栏提示与日志共用,别在两处各写一份)。"""
|
| 83 |
+
if spec not in settings.GRIPPER_FORCE_MATRIX_SPECS:
|
| 84 |
+
return tr("未知规格 {}", spec)
|
| 85 |
+
scale = settings.GRIPPER_FORCE_MATRIX_SCALES.get(spec)
|
| 86 |
+
if scale is None:
|
| 87 |
+
return tr("float32 原值直存(无截断无饱和,体积最大)")
|
| 88 |
+
if scale == 1:
|
| 89 |
+
return tr("int16 行差分量化(1 mN 台阶、向零截断,体积最小)")
|
| 90 |
+
return tr("int16 定标 ×{}(分辨率 {} mN,饱和上限 ±{} mN)",
|
| 91 |
+
scale, 1.0 / scale, 32767.0 / scale)
|
collector/core/pipeline.py
CHANGED
|
@@ -166,6 +166,11 @@ class CameraPipeline(QObject):
|
|
| 166 |
# SLAM 轨迹点按帧窗口累积(每点 8 值 [t,x,y,z,qx,qy,qz,qw] 扁平),
|
| 167 |
# 行写入时并入 snapshot——轨迹 txt 侧车由此废弃(并入 episode parquet)
|
| 168 |
self._gripper_traj: Dict[str, list] = {}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 169 |
# 夹爪非视频槽设备 ID(pose/force 槽不走 register_external_source,
|
| 170 |
# 单独进 status 列)
|
| 171 |
self._gripper_device_ids: set = set()
|
|
@@ -589,6 +594,7 @@ class CameraPipeline(QObject):
|
|
| 589 |
with self._gripper_snapshots_lock:
|
| 590 |
self._gripper_snapshots.clear()
|
| 591 |
self._gripper_traj.clear()
|
|
|
|
| 592 |
|
| 593 |
# B+ 方案: 启动独立写入线程(精确 30fps,完全不受主线程 UI 影响)
|
| 594 |
self._write_thread = threading.Thread(
|
|
@@ -836,7 +842,8 @@ class CameraPipeline(QObject):
|
|
| 836 |
with self._gripper_snapshots_lock:
|
| 837 |
self._gripper_snapshots[key] = value
|
| 838 |
|
| 839 |
-
def write_slam_trajectory(self, pose7, prefix: str = "", timestamp=None
|
|
|
|
| 840 |
"""SLAM 轨迹点 [t,x,y,z,qx,qy,qz,qw] 累积(P3;桥接 pose_ready 回调)。
|
| 841 |
|
| 842 |
prefix 为双夹爪命名空间:rig1 空串(旧键 slam_trajectory 不变),
|
|
@@ -847,6 +854,13 @@ class CameraPipeline(QObject):
|
|
| 847 |
缺位姿的行会被填成 [0]*7 被下游当真实位姿读;轨迹点带真实时间戳、
|
| 848 |
信息严格更多,是 SLAM 位姿的唯一落盘形态。轨迹不再单独落 txt
|
| 849 |
侧车,全部信息都在一条 episode parquet 里。
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 850 |
"""
|
| 851 |
if pose7 is None:
|
| 852 |
return
|
|
@@ -860,11 +874,21 @@ class CameraPipeline(QObject):
|
|
| 860 |
return
|
| 861 |
if not np.isfinite(t):
|
| 862 |
return
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 863 |
with self._gripper_snapshots_lock:
|
| 864 |
if self._writer is None or not self._recording:
|
| 865 |
return
|
| 866 |
-
|
| 867 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 868 |
|
| 869 |
def write_gripper_state(self, state3, prefix: str = ""):
|
| 870 |
"""夹爪状态快照 [open_pct, gripped, fz_mn](P4;GripState 输出)。"""
|
|
@@ -934,7 +958,8 @@ class CameraPipeline(QObject):
|
|
| 934 |
"""取走并清空本帧窗口内的夹爪快照(写入线程调用)。
|
| 935 |
|
| 936 |
轨迹点在此并入:{prefix}slam_trajectory = 本窗口内全部点的
|
| 937 |
-
扁平 8 值列表(无点则不写键)
|
|
|
|
| 938 |
"""
|
| 939 |
with self._gripper_snapshots_lock:
|
| 940 |
snapshot = dict(self._gripper_snapshots)
|
|
@@ -942,7 +967,15 @@ class CameraPipeline(QObject):
|
|
| 942 |
for key, points in self._gripper_traj.items():
|
| 943 |
if points:
|
| 944 |
snapshot[key] = points
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 945 |
self._gripper_traj.clear()
|
|
|
|
| 946 |
return snapshot
|
| 947 |
|
| 948 |
def _pop_glove_keypoints(self) -> Dict[str, np.ndarray]:
|
|
|
|
| 166 |
# SLAM 轨迹点按帧窗口累积(每点 8 值 [t,x,y,z,qx,qy,qz,qw] 扁平),
|
| 167 |
# 行写入时并入 snapshot——轨迹 txt 侧车由此废弃(并入 episode parquet)
|
| 168 |
self._gripper_traj: Dict[str, list] = {}
|
| 169 |
+
# 与 _gripper_traj 逐点同序并行的宿主时刻(int64 纳秒)。**必须与
|
| 170 |
+
# 点列表锁步**:每点恒定追加一个值,无戳时补 0(= 未知,与
|
| 171 |
+
# hardware_ns / *_force_ns 的 0 约定一致),这样两列长度恒相等、
|
| 172 |
+
# 下标即配对。缺戳的点若跳过不追加,两列就会错位且无法察觉。
|
| 173 |
+
self._gripper_traj_ns: Dict[str, list] = {}
|
| 174 |
# 夹爪非视频槽设备 ID(pose/force 槽不走 register_external_source,
|
| 175 |
# 单独进 status 列)
|
| 176 |
self._gripper_device_ids: set = set()
|
|
|
|
| 594 |
with self._gripper_snapshots_lock:
|
| 595 |
self._gripper_snapshots.clear()
|
| 596 |
self._gripper_traj.clear()
|
| 597 |
+
self._gripper_traj_ns.clear()
|
| 598 |
|
| 599 |
# B+ 方案: 启动独立写入线程(精确 30fps,完全不受主线程 UI 影响)
|
| 600 |
self._write_thread = threading.Thread(
|
|
|
|
| 842 |
with self._gripper_snapshots_lock:
|
| 843 |
self._gripper_snapshots[key] = value
|
| 844 |
|
| 845 |
+
def write_slam_trajectory(self, pose7, prefix: str = "", timestamp=None,
|
| 846 |
+
host_ns=None):
|
| 847 |
"""SLAM 轨迹点 [t,x,y,z,qx,qy,qz,qw] 累积(P3;桥接 pose_ready 回调)。
|
| 848 |
|
| 849 |
prefix 为双夹爪命名空间:rig1 空串(旧键 slam_trajectory 不变),
|
|
|
|
| 854 |
缺位姿的行会被填成 [0]*7 被下游当真实位姿读;轨迹点带真实时间戳、
|
| 855 |
信息严格更多,是 SLAM 位姿的唯一落盘形态。轨迹不再单独落 txt
|
| 856 |
侧车,全部信息都在一条 episode parquet 里。
|
| 857 |
+
|
| 858 |
+
host_ns:该取样帧的宿主单调钟纳秒(native 侧 CLOCK_MONOTONIC,
|
| 859 |
+
与 `hardware_ns` 同时基),逐点并行落成
|
| 860 |
+
`{prefix}slam_trajectory_ns`。**t 与 host_ns 不同源**:t 是相机
|
| 861 |
+
传感器钟,换算到宿主钟要一个只能拟合的偏移(实测三个反推估计
|
| 862 |
+
散布 287607.586/.637/.690,跨度 ~104ms,已超一个帧间隔),所以
|
| 863 |
+
没有 host_ns 就只能靠行号配 slam 点与视频帧。0/None = 无戳。
|
| 864 |
"""
|
| 865 |
if pose7 is None:
|
| 866 |
return
|
|
|
|
| 874 |
return
|
| 875 |
if not np.isfinite(t):
|
| 876 |
return
|
| 877 |
+
try:
|
| 878 |
+
host_value = int(host_ns) if host_ns is not None else 0
|
| 879 |
+
except (TypeError, ValueError, OverflowError):
|
| 880 |
+
host_value = 0
|
| 881 |
+
if host_value < 0:
|
| 882 |
+
host_value = 0
|
| 883 |
with self._gripper_snapshots_lock:
|
| 884 |
if self._writer is None or not self._recording:
|
| 885 |
return
|
| 886 |
+
key = f"{prefix}slam_trajectory"
|
| 887 |
+
self._gripper_traj.setdefault(key, []).extend([t, *values])
|
| 888 |
+
# 与点列表**锁步**:每点恒定追加一个值(无戳补 0),
|
| 889 |
+
# 两列长度恒等、下标即配对。缺戳就跳过不追加会让两列
|
| 890 |
+
# 错位且下游无从察觉。
|
| 891 |
+
self._gripper_traj_ns.setdefault(key, []).append(host_value)
|
| 892 |
|
| 893 |
def write_gripper_state(self, state3, prefix: str = ""):
|
| 894 |
"""夹爪状态快照 [open_pct, gripped, fz_mn](P4;GripState 输出)。"""
|
|
|
|
| 958 |
"""取走并清空本帧窗口内的夹爪快照(写入线程调用)。
|
| 959 |
|
| 960 |
轨迹点在此并入:{prefix}slam_trajectory = 本窗口内全部点的
|
| 961 |
+
扁平 8 值列表(无点则不写键);其宿主时刻同序并行落
|
| 962 |
+
{prefix}slam_trajectory_ns(int64 纳秒,逐点对齐)。
|
| 963 |
"""
|
| 964 |
with self._gripper_snapshots_lock:
|
| 965 |
snapshot = dict(self._gripper_snapshots)
|
|
|
|
| 967 |
for key, points in self._gripper_traj.items():
|
| 968 |
if points:
|
| 969 |
snapshot[key] = points
|
| 970 |
+
stamps = self._gripper_traj_ns.get(key) or []
|
| 971 |
+
# 长度必须是点数(点=8 值)。全 0 表示本段所有点都
|
| 972 |
+
# 没戳(旧 native 二进制不打印 Host 字段)——此时
|
| 973 |
+
# 不建列,免得下游拿到一列「看着有、其实全未知」的
|
| 974 |
+
# 时刻而误以为可对齐。
|
| 975 |
+
if len(stamps) == len(points) // 8 and any(stamps):
|
| 976 |
+
snapshot[f"{key}_ns"] = stamps
|
| 977 |
self._gripper_traj.clear()
|
| 978 |
+
self._gripper_traj_ns.clear()
|
| 979 |
return snapshot
|
| 980 |
|
| 981 |
def _pop_glove_keypoints(self) -> Dict[str, np.ndarray]:
|
collector/docs/data.md
CHANGED
|
@@ -302,6 +302,7 @@ data/recordings/ # 录制根目录(settings.RECORDING
|
|
| 302 |
| `observation.gripper_{left,right}_force_matrix` | list<int16> / list<float32> | 250×250×3 力矩阵(档位见下) |
|
| 303 |
| `observation.gripper_{left,right}_force_matrix_ns` | int64 | 该力矩阵样本的采集时刻 |
|
| 304 |
| `observation.slam_trajectory` | list<double> | 轨迹点 `[t,x,y,z,qx,qy,qz,qw]`×N,变长 |
|
|
|
|
| 305 |
| `observation.gripper_state` | list<float32, 3> | `[open_pct, gripped, fz_mn]` |
|
| 306 |
|
| 307 |
力矩阵档位记在**每段一份**的 `meta/episodes` 行(`force_matrix_specs` 列),
|
|
@@ -337,6 +338,42 @@ mapping, residual_ns = align_episode("episode-000.parquet", side="left")
|
|
| 337 |
的 `hardware_ns` 还被 `pyqtSignal(int)` 截成 32 位(每 2.147 s 翻符号)。
|
| 338 |
脚本对这类 episode 明确报「无法对齐」而不是拿行号或增长率去猜。
|
| 339 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 340 |
### 4.6 `meta/info.json` 字段概览(LeRobot v3 兼容,上传服务器严格依赖)
|
| 341 |
|
| 342 |
| 键 | 含义 |
|
|
|
|
| 302 |
| `observation.gripper_{left,right}_force_matrix` | list<int16> / list<float32> | 250×250×3 力矩阵(档位见下) |
|
| 303 |
| `observation.gripper_{left,right}_force_matrix_ns` | int64 | 该力矩阵样本的采集时刻 |
|
| 304 |
| `observation.slam_trajectory` | list<double> | 轨迹点 `[t,x,y,z,qx,qy,qz,qw]`×N,变长 |
|
| 305 |
+
| `observation.slam_trajectory_ns` | list<int64> | 每个轨迹点对应取样帧的**宿主时刻**,与 `slam_trajectory` 同序等长 |
|
| 306 |
| `observation.gripper_state` | list<float32, 3> | `[open_pct, gripped, fz_mn]` |
|
| 307 |
|
| 308 |
力矩阵档位记在**每段一份**的 `meta/episodes` 行(`force_matrix_specs` 列),
|
|
|
|
| 338 |
的 `hardware_ns` 还被 `pyqtSignal(int)` 截成 32 位(每 2.147 s 翻符号)。
|
| 339 |
脚本对这类 episode 明确报「无法对齐」而不是拿行号或增长率去猜。
|
| 340 |
|
| 341 |
+
### 4.5.2 SLAM 点与视频帧的时间对齐
|
| 342 |
+
|
| 343 |
+
`slam_trajectory` 每点的第 0 个字段 `t` 是**相机传感器钟**上的时间(native
|
| 344 |
+
侧 `img->timestamp`,相对按下 ENTER 设原点那一刻)。它与 RGB 帧的
|
| 345 |
+
`hardware_ns`(宿主单调钟)**不同源**,两者之间只差一个近似常量但带抖动的
|
| 346 |
+
偏移——只能从数据反推、不能换算。实测同一段数据反推出的偏移有三个估计值
|
| 347 |
+
(`arrival_monotonic − t` 中位、行桶中位、逐点拟合最优),散布约 **104 ms**,
|
| 348 |
+
已超一个帧间隔(33.3 ms)。
|
| 349 |
+
|
| 350 |
+
因此 v1.3.6 起 native 在每个取样帧进进程时取一次 `CLOCK_MONOTONIC`
|
| 351 |
+
(`RawImageFrame.host_mono_ns`),随帧走完预处理/跟踪/排队全程,最终与位姿
|
| 352 |
+
一起打在 stdout 的 `Host:(<ns>)` 字段上,落成
|
| 353 |
+
`observation.{prefix}slam_trajectory_ns`。
|
| 354 |
+
|
| 355 |
+
> **改这段 native 代码要认准文件**:构建源是
|
| 356 |
+
> `online/ORB-SLAM/Examples/fays/fayssense_orb_slam.cc`(由
|
| 357 |
+
> `online/dist/fays_opencv48/CMakeLists.txt` 的 `FAYS_BRIDGE_SOURCE` 硬指向)。
|
| 358 |
+
> `core/gripper/native/ORB-SLAM/Examples/fays/` 下那份是**陈旧副本,不是构建源**,
|
| 359 |
+
> 改它不会进二进制——且它缺少 2026-09-11 部署的崩溃修复。两棵树都被
|
| 360 |
+
> `.gitignore`(`online/` 第 58 行、`core/gripper/native/` 第 62 行),桥接源码
|
| 361 |
+
> 没有版本控制兜底,改完必须重编二进制才会生效。
|
| 362 |
+
|
| 363 |
+
- **与 `slam_trajectory` 同序等长**,下标即配对,所以某帧有 N 个轨迹点就有 N
|
| 364 |
+
个时刻。这要求时刻列表**锁步**累积:某个点没有戳时补 `0`(= 未知,与
|
| 365 |
+
`*_force_ns` 的约定一致)而**不是跳过**,跳过会让后续点全部错位且无从察觉。
|
| 366 |
+
- 有了它,配 slam 点↔视频帧退化成一次 `searchsorted`:按 `hardware_ns` 找
|
| 367 |
+
最近的 `slam_trajectory_ns`。**不需要拟合偏移,更不需要插值**——从实测点里
|
| 368 |
+
取最近邻是合法的 1:1,不是造假数据。
|
| 369 |
+
- **全段无戳时不建该列**(例如 Python 侧已升级但 native 二进制未重编)。
|
| 370 |
+
下游按列是否存在判断「本段能否按时间对齐」;列存在但个别值为 0 表示那
|
| 371 |
+
一个点缺戳。
|
| 372 |
+
- **行数不等于点数。** 写线程在落后超过一帧时会重置时钟并**跳过**那些 tick
|
| 373 |
+
(`core/pipeline.py` 的 `_write_loop`),实测一段 11.4 s 的录制因此少写
|
| 374 |
+
11 行;slam 点侧则约丢 0.22%。所以「slam 点数 == 视频帧数」不成立,
|
| 375 |
+
按时间戳配对后剩下未配对的点/帧是正常现象,不是数据损坏。
|
| 376 |
+
|
| 377 |
### 4.6 `meta/info.json` 字段概览(LeRobot v3 兼容,上传服务器严格依赖)
|
| 378 |
|
| 379 |
| 键 | 含义 |
|
collector/scripts/pack_lite.py
CHANGED
|
@@ -1,28 +1,42 @@
|
|
| 1 |
"""极简采集版发布包收集脚本 —— 把 lite 运行时闭包收集到一个目录。
|
| 2 |
|
| 3 |
-
只收集 start_lite.bat 真正需要的最小集合(约 1.
|
| 4 |
-
不拷贝 venv/venv_lite/data/wheels
|
| 5 |
-
demo/其它测试、
|
|
|
|
| 6 |
|
| 7 |
-
代码文件以仓库根为源;start_lite.bat/.sh 与 使用说明_lite.md 是分发壳
|
| 8 |
-
|
|
|
|
|
|
|
|
|
|
| 9 |
|
| 10 |
--force 只重建代码部分(main_lite/requirements-lite/config/core/ui/tools),
|
| 11 |
-
venv_lite/、wheels/、data/
|
| 12 |
-
"发布模板"和"本机测试部署目录"用
|
| 13 |
-
不会被冲掉)。
|
| 14 |
|
| 15 |
-
收集完成后自动做导入自检:在目标目录内 `import main_lite`,
|
| 16 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
|
| 18 |
闭包清单与 tools/tests/lite_import_guard_test.py 的 WHITELIST 一致,
|
| 19 |
外加 egodata_writer/d435_camera 实际 import 的 core.calibration。
|
| 20 |
|
| 21 |
用法:
|
| 22 |
-
python scripts/pack_lite.py 收集到 lite_package/
|
|
|
|
| 23 |
python scripts/pack_lite.py --out D:/lite 指定目录
|
| 24 |
python scripts/pack_lite.py --with-wheels 连同 wheels/ 离线依赖包一起
|
| 25 |
-
python scripts/pack_lite.py --force 重建代码部分(保留环境)
|
| 26 |
"""
|
| 27 |
import argparse
|
| 28 |
import os
|
|
@@ -32,7 +46,7 @@ import sys
|
|
| 32 |
|
| 33 |
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
| 34 |
|
| 35 |
-
# 分发壳的
|
| 36 |
DIST_SOURCE = os.path.join(ROOT, "lite_package")
|
| 37 |
|
| 38 |
# 代码文件(仓库根为源,收集时拷入包目录)
|
|
@@ -41,17 +55,82 @@ CODE_FILES = [
|
|
| 41 |
"requirements-lite.txt",
|
| 42 |
]
|
| 43 |
|
| 44 |
-
# 分发壳文件(
|
| 45 |
DIST_FILES = [
|
| 46 |
"start_lite.bat",
|
| 47 |
"start_lite.sh",
|
| 48 |
"使用说明_lite.md",
|
| 49 |
]
|
| 50 |
|
| 51 |
-
|
| 52 |
-
|
| 53 |
-
"
|
| 54 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 55 |
|
| 56 |
# 单文件拷贝:目录 → 文件列表(只收 lite 闭包,主程序 UI/重依赖模块不带)
|
| 57 |
MODULE_FILES = {
|
|
@@ -66,6 +145,11 @@ MODULE_FILES = {
|
|
| 66 |
"api_client.py", "database.py", "helpers.py",
|
| 67 |
# egodata_writer.py:41 / d435_camera.py:447 实际 import
|
| 68 |
"calibration.py",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 69 |
],
|
| 70 |
}
|
| 71 |
|
|
@@ -73,26 +157,228 @@ MODULE_FILES = {
|
|
| 73 |
SELFTEST_FILES = [
|
| 74 |
"tools/tests/lite_import_guard_test.py",
|
| 75 |
"tools/tests/lite_smoke_test.py",
|
|
|
|
| 76 |
]
|
| 77 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 78 |
|
| 79 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 80 |
out_dir = os.path.abspath(out_dir)
|
| 81 |
if force:
|
| 82 |
-
# 只重建代码路径;venv_lite/wheels/data 与分发壳原样保留
|
|
|
|
|
|
|
|
|
|
| 83 |
for rel in (CODE_FILES + ["config", "core", "ui", "tools"]):
|
| 84 |
-
|
| 85 |
-
if os.path.isdir(p):
|
| 86 |
-
shutil.rmtree(p, ignore_errors=True)
|
| 87 |
-
elif os.path.isfile(p):
|
| 88 |
-
os.remove(p)
|
| 89 |
elif os.path.isfile(os.path.join(out_dir, "main_lite.py")):
|
| 90 |
print(f"[跳过] 目标目录已存在: {out_dir}")
|
| 91 |
-
print(" --force 只重建代码部分,venv_lite/wheels/data 会保留")
|
| 92 |
sys.exit(0)
|
| 93 |
|
| 94 |
copied = []
|
| 95 |
-
for rel in CODE_FILES + SELFTEST_FILES
|
|
|
|
| 96 |
src = os.path.join(ROOT, rel)
|
| 97 |
if not os.path.isfile(src):
|
| 98 |
print(f"[警告] 缺文件: {rel}")
|
|
@@ -103,13 +389,13 @@ def collect(out_dir: str, with_wheels: bool, force: bool) -> None:
|
|
| 103 |
copied.append(rel)
|
| 104 |
|
| 105 |
for rel in DIST_FILES:
|
| 106 |
-
src =
|
|
|
|
|
|
|
|
|
|
| 107 |
dst = os.path.join(out_dir, rel)
|
| 108 |
if os.path.abspath(src) == os.path.abspath(dst):
|
| 109 |
-
copied.append(rel) # 源
|
| 110 |
-
continue
|
| 111 |
-
if not os.path.isfile(src):
|
| 112 |
-
print(f"[警告] 缺分发文件: {rel}(应在 lite_package/ 下)")
|
| 113 |
continue
|
| 114 |
shutil.copy2(src, dst)
|
| 115 |
copied.append(rel)
|
|
@@ -117,7 +403,18 @@ def collect(out_dir: str, with_wheels: bool, force: bool) -> None:
|
|
| 117 |
for rel in WHOLE_DIRS:
|
| 118 |
src = os.path.join(ROOT, rel)
|
| 119 |
dst = os.path.join(out_dir, rel)
|
| 120 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 121 |
copied.append(rel + "/")
|
| 122 |
|
| 123 |
for sub, files in MODULE_FILES.items():
|
|
@@ -151,25 +448,40 @@ def collect(out_dir: str, with_wheels: bool, force: bool) -> None:
|
|
| 151 |
print("[失败] 目标目录内 import main_lite 自检未通过:")
|
| 152 |
print(proc.stderr.decode("utf-8", "replace"))
|
| 153 |
sys.exit(1)
|
|
|
|
| 154 |
|
| 155 |
total = sum(os.path.getsize(os.path.join(out_dir, r))
|
| 156 |
for r in copied if os.path.isfile(os.path.join(out_dir, r)))
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 157 |
print(f"[完成] 收集 {len(copied)} 个文件到 {out_dir}/,"
|
| 158 |
-
f"代码共 {total / 1024:.0f} KB,import main_lite 自检通过")
|
| 159 |
-
print(
|
| 160 |
-
+ (" + wheels/" if with_wheels else ""))
|
| 161 |
|
| 162 |
|
| 163 |
def main() -> None:
|
| 164 |
parser = argparse.ArgumentParser(description="收集极简采集版发布包")
|
| 165 |
parser.add_argument("--out", default=os.path.join(ROOT, "lite_package"),
|
| 166 |
help="输出目录(默认仓库根 lite_package/)")
|
|
|
|
|
|
|
|
|
|
|
|
|
| 167 |
parser.add_argument("--with-wheels", action="store_true",
|
| 168 |
help="连同 wheels/ 离线依赖包一起收集")
|
| 169 |
parser.add_argument("--force", action="store_true",
|
| 170 |
-
help="重建代码部分(保留 venv_lite/wheels/data)")
|
| 171 |
args = parser.parse_args()
|
| 172 |
-
|
|
|
|
| 173 |
|
| 174 |
|
| 175 |
if __name__ == "__main__":
|
|
|
|
| 1 |
"""极简采集版发布包收集脚本 —— 把 lite 运行时闭包收集到一个目录。
|
| 2 |
|
| 3 |
+
只收集 start_lite.bat 真正需要的最小集合(约 1.6 MB 代码)与夹爪原生
|
| 4 |
+
载荷(Linux 包,约 440 MB),不拷贝 venv/venv_lite/data/wheels
|
| 5 |
+
(--with-wheels 除外)、tools 下的 demo/其它测试、
|
| 6 |
+
stouch_glove_toolkit-*/ 等与 lite 无关的内容。
|
| 7 |
|
| 8 |
+
代码文件以仓库根为源;start_lite.bat/.sh 与 使用说明_lite.md 是分发壳。
|
| 9 |
+
2026-09-16 起三个分发壳全部住在**仓库根**(与 main_lite.py、
|
| 10 |
+
requirements-lite.txt 同级 —— 这几个齐了根目录本身就是可直接运行的 lite
|
| 11 |
+
部署目录,根上已有 venv_lite/),由 _dist_shell_source 优先从那里取,
|
| 12 |
+
复制进包目录。
|
| 13 |
|
| 14 |
--force 只重建代码部分(main_lite/requirements-lite/config/core/ui/tools),
|
| 15 |
+
venv_lite/、wheels/、data/、分发壳与 core/gripper/native(470MB 载荷)
|
| 16 |
+
原样保留 —— lite_package 可同时当"发布模板"和"本机测试部署目录"用
|
| 17 |
+
(在包内跑过 start_lite.sh 装的环境不会被冲掉)。
|
| 18 |
|
| 19 |
+
收集完成后自动做导入自检:在目标目录内 `import main_lite`,缺任何被闭包
|
| 20 |
+
引用的模块都会当场报错(与 start_lite.bat 的错误 E 同口径);Linux 包再
|
| 21 |
+
断言夹爪闭包(`core.gripper.bridge`)与 `gripper_resources_available()`。
|
| 22 |
+
|
| 23 |
+
两个目标平台收的**夹爪 Python 代码是同一份**(core/gripper/**、core/
|
| 24 |
+
gripper_codec.py、core/s80m_manager.py),差别只在平台绑定件:Windows 包
|
| 25 |
+
不收 core/gripper/native/(ELF 原生栈)与 core/gripper/sightac_sdk/(pyarmor
|
| 26 |
+
.so 与 libSonixCamera.so 也是 ELF),并硬断言包内零 ELF。缺这些时
|
| 27 |
+
gripper_resources_available() 为假 → 扫描不到夹爪(设备组框还在、里面是
|
| 28 |
+
空的,属预期降级)。于是
|
| 29 |
+
「Windows 包」≡「Linux 包去掉载荷」,后者在 Linux 开发机上可直接造出来验证。
|
| 30 |
|
| 31 |
闭包清单与 tools/tests/lite_import_guard_test.py 的 WHITELIST 一致,
|
| 32 |
外加 egodata_writer/d435_camera 实际 import 的 core.calibration。
|
| 33 |
|
| 34 |
用法:
|
| 35 |
+
python scripts/pack_lite.py 收集到 lite_package/(本机平台)
|
| 36 |
+
python scripts/pack_lite.py --target windows 收集 Windows 包(代码同源,不含平台绑定件)
|
| 37 |
python scripts/pack_lite.py --out D:/lite 指定目录
|
| 38 |
python scripts/pack_lite.py --with-wheels 连同 wheels/ 离线依赖包一起
|
| 39 |
+
python scripts/pack_lite.py --force 重建代码部分(保留环境与载荷)
|
| 40 |
"""
|
| 41 |
import argparse
|
| 42 |
import os
|
|
|
|
| 46 |
|
| 47 |
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
| 48 |
|
| 49 |
+
# 分发壳的兜底目录(三个壳 2026-09-16 都搬到仓库根了,这里只作回落)
|
| 50 |
DIST_SOURCE = os.path.join(ROOT, "lite_package")
|
| 51 |
|
| 52 |
# 代码文件(仓库根为源,收集时拷入包目录)
|
|
|
|
| 55 |
"requirements-lite.txt",
|
| 56 |
]
|
| 57 |
|
| 58 |
+
# 分发壳文件(仓库根优先,找不到再回落 lite_package/;均原样复制)
|
| 59 |
DIST_FILES = [
|
| 60 |
"start_lite.bat",
|
| 61 |
"start_lite.sh",
|
| 62 |
"使用说明_lite.md",
|
| 63 |
]
|
| 64 |
|
| 65 |
+
|
| 66 |
+
def _dist_shell_source(rel: str):
|
| 67 |
+
"""分发壳的源路径:仓库根优先,其次 lite_package/;都没有返回 None。
|
| 68 |
+
|
| 69 |
+
三个壳 2026-09-16 都搬到了仓库根(与 main_lite.py 同级,使根目录直接
|
| 70 |
+
可运行);lite_package/ 只作回落,服务老布局的目录。
|
| 71 |
+
"""
|
| 72 |
+
for base in (ROOT, DIST_SOURCE):
|
| 73 |
+
path = os.path.join(base, rel)
|
| 74 |
+
if os.path.isfile(path):
|
| 75 |
+
return path
|
| 76 |
+
return None
|
| 77 |
+
|
| 78 |
+
|
| 79 |
+
# 夹爪载荷:Linux 包整目录收集,Windows 包只收代码(见 _PLATFORM_BOUND)。
|
| 80 |
+
GRIPPER_REL = "core/gripper"
|
| 81 |
+
GRIPPER_PAYLOAD_REL = "core/gripper/native" # --force 时原��保留
|
| 82 |
+
|
| 83 |
+
# 与平台绑定的原生件:Windows 包一律不收。
|
| 84 |
+
# native/ —— 整套 ELF x86-64 原生栈(460MB)
|
| 85 |
+
# sightac_sdk/ —— pyarmor 运行时 .so 与 libSonixCamera.so 同样是 ELF
|
| 86 |
+
# 缺它们时 paths.gripper_resources_available() 为假 → 扫描不到夹爪(设备组
|
| 87 |
+
# 框照建、里面空着),
|
| 88 |
+
# 且 core/gripper/*.py 在 Windows 上因顶层 `import fcntl` 本就导入不了
|
| 89 |
+
# (双重保险)。**Python 代码两个包都收**:一份源码两个包,差异只在载荷,
|
| 90 |
+
# 于是「Windows 包」= 「Linux 包去掉载荷」,这种形态在 Linux 开发机上可以
|
| 91 |
+
# 直接造出来验证(mv 走 native/ 即可),不必真去 Windows 上撞。
|
| 92 |
+
_PLATFORM_BOUND_NAMES = ("native", "sightac_sdk")
|
| 93 |
+
|
| 94 |
+
# 载荷里可以剪掉的(≈33MB):备份二进制、静态库、缓存。
|
| 95 |
+
# ★ 绝不能粗剪:native/ORB-SLAM 必须留着 —— 桥接二进制的 RUNPATH 里有
|
| 96 |
+
# $ORIGIN/../../../ORB-SLAM/Thirdparty/{DBoW2,g2o}/lib(实测 readelf),
|
| 97 |
+
# 剪掉会让 libDBoW2/libg2o 在客户机上找不到;ORBvoc.txt(139MB)与
|
| 98 |
+
# FaysSense_VI_Kit_Release/ 同理全是运行时资产。
|
| 99 |
+
_PAYLOAD_SKIP_SUFFIX = (".pyc", ".a")
|
| 100 |
+
_PAYLOAD_SKIP_INFIX = (".pre_", ".bak_") # 现场备份:xxx.pre_rebuild_2026...
|
| 101 |
+
_PAYLOAD_SKIP_NAMES = ("__pycache__", "camera_service_src")
|
| 102 |
+
|
| 103 |
+
|
| 104 |
+
def _gripper_ignore_factory(src_root: str, windows: bool,
|
| 105 |
+
keep_existing_native: bool):
|
| 106 |
+
"""core/gripper 整目录收集的排除表(见上方注释)。
|
| 107 |
+
|
| 108 |
+
windows: 连平台绑定件一起排(Windows 包)
|
| 109 |
+
keep_existing_native: 目标包已有 native/(--force 重打包)时整棵跳过 ——
|
| 110 |
+
copytree 遇到已存在的同名软链会直接抛 File exists(symlinks=True 下
|
| 111 |
+
的 os.symlink 不覆盖),实测第二次打包必炸。
|
| 112 |
+
"""
|
| 113 |
+
def _ignore(dir_path: str, names: list):
|
| 114 |
+
ignored = set()
|
| 115 |
+
for name in names:
|
| 116 |
+
if name in _PAYLOAD_SKIP_NAMES:
|
| 117 |
+
ignored.add(name)
|
| 118 |
+
elif name.endswith(_PAYLOAD_SKIP_SUFFIX) or any(
|
| 119 |
+
s in name for s in _PAYLOAD_SKIP_INFIX):
|
| 120 |
+
ignored.add(name)
|
| 121 |
+
if os.path.abspath(dir_path) == os.path.abspath(src_root):
|
| 122 |
+
if windows:
|
| 123 |
+
ignored.update(_PLATFORM_BOUND_NAMES)
|
| 124 |
+
if keep_existing_native and not windows:
|
| 125 |
+
ignored.add("native")
|
| 126 |
+
return ignored
|
| 127 |
+
return _ignore
|
| 128 |
+
|
| 129 |
+
|
| 130 |
+
# 整目录拷贝(core.glove_usb 是包,内部有 __init__.py 与互引模块)。
|
| 131 |
+
# core.gripper 的排除表按目标平台现算(见 _gripper_ignore_factory),
|
| 132 |
+
# 且必须 symlinks=True —— 载荷里有 76 个相对软链,展开会让体积 468MB→800MB。
|
| 133 |
+
WHOLE_DIRS = ["core/glove_usb", GRIPPER_REL]
|
| 134 |
|
| 135 |
# 单文件拷贝:目录 → 文件列表(只收 lite 闭包,主程序 UI/重依赖模块不带)
|
| 136 |
MODULE_FILES = {
|
|
|
|
| 145 |
"api_client.py", "database.py", "helpers.py",
|
| 146 |
# egodata_writer.py:41 / d435_camera.py:447 实际 import
|
| 147 |
"calibration.py",
|
| 148 |
+
# 夹爪闭包(2026-09-16 并入 lite):力矩阵编码的家(主程序
|
| 149 |
+
# ui.main_window 只是 re-export —— 契约不能存两份)
|
| 150 |
+
"gripper_codec.py",
|
| 151 |
+
# core/gripper/bridge.py:39 顶层 import 它的双目丢帧窗口常量
|
| 152 |
+
"s80m_manager.py",
|
| 153 |
],
|
| 154 |
}
|
| 155 |
|
|
|
|
| 157 |
SELFTEST_FILES = [
|
| 158 |
"tools/tests/lite_import_guard_test.py",
|
| 159 |
"tools/tests/lite_smoke_test.py",
|
| 160 |
+
"tools/tests/lite_gripper_smoke_test.py",
|
| 161 |
]
|
| 162 |
|
| 163 |
+
# 随包工具:只有 Linux 包带 —— 它要驱动 native/ 里的厂商二进制,
|
| 164 |
+
# 在 Windows 上无处落脚。夹爪的「重新读取出厂标定」在 lite 里**不做 UI**,
|
| 165 |
+
# 命令行就是唯一入口,所以这个工具必须在包里,否则那条路在纯 lite 机器上
|
| 166 |
+
# 走不通(它顶层只 import stdlib + core.gripper.*,venv_lite 的依赖足够)。
|
| 167 |
+
TOOL_FILES_LINUX = [
|
| 168 |
+
"tools/import_gripper_calibration.py",
|
| 169 |
+
]
|
| 170 |
|
| 171 |
+
|
| 172 |
+
def _force_remove(path: str, keep: str) -> None:
|
| 173 |
+
"""删除 path;但 keep 那棵树(连同其本身)一个字节都不动。"""
|
| 174 |
+
path, keep = os.path.abspath(path), os.path.abspath(keep)
|
| 175 |
+
if path == keep:
|
| 176 |
+
return # 保留整棵树(夹爪载荷)
|
| 177 |
+
if keep.startswith(path + os.sep):
|
| 178 |
+
if os.path.isdir(path): # path 是 keep 的祖先:只删其它子项
|
| 179 |
+
for name in os.listdir(path):
|
| 180 |
+
_force_remove(os.path.join(path, name), keep)
|
| 181 |
+
return
|
| 182 |
+
if os.path.isdir(path):
|
| 183 |
+
shutil.rmtree(path, ignore_errors=True)
|
| 184 |
+
elif os.path.isfile(path):
|
| 185 |
+
os.remove(path)
|
| 186 |
+
|
| 187 |
+
|
| 188 |
+
def _tree_size(path: str) -> int:
|
| 189 |
+
"""目录的字节数(不跟随软链 —— 与分发后的实际落盘一致)。"""
|
| 190 |
+
total = 0
|
| 191 |
+
for dir_path, _, files in os.walk(path):
|
| 192 |
+
for name in files:
|
| 193 |
+
full = os.path.join(dir_path, name)
|
| 194 |
+
if not os.path.islink(full):
|
| 195 |
+
try:
|
| 196 |
+
total += os.path.getsize(full)
|
| 197 |
+
except OSError:
|
| 198 |
+
pass
|
| 199 |
+
return total
|
| 200 |
+
|
| 201 |
+
|
| 202 |
+
def _count_symlinks(path: str) -> int:
|
| 203 |
+
count = 0
|
| 204 |
+
for dir_path, _, files in os.walk(path):
|
| 205 |
+
for name in files:
|
| 206 |
+
if os.path.islink(os.path.join(dir_path, name)):
|
| 207 |
+
count += 1
|
| 208 |
+
return count
|
| 209 |
+
|
| 210 |
+
|
| 211 |
+
def _count_py(path: str) -> int:
|
| 212 |
+
"""**.py 个数(不跟随软链、不计 __pycache__)—— Windows 包「同一份源码」
|
| 213 |
+
的量化证据:与 Linux 包只差 sightac_sdk/ 里那 7 个包壳文件。"""
|
| 214 |
+
count = 0
|
| 215 |
+
for dir_path, dirs, files in os.walk(path):
|
| 216 |
+
dirs[:] = [d for d in dirs if d != "__pycache__"]
|
| 217 |
+
count += sum(1 for n in files
|
| 218 |
+
if n.endswith(".py")
|
| 219 |
+
and not os.path.islink(os.path.join(dir_path, n)))
|
| 220 |
+
return count
|
| 221 |
+
|
| 222 |
+
|
| 223 |
+
def _find_elf(root: str) -> list:
|
| 224 |
+
"""root 下所有 ELF 文件(前 4 字节 \\x7fELF)的相对路径。
|
| 225 |
+
|
| 226 |
+
Windows 包的硬不变量就是「一个 ELF 都没有」—— native/ 与 sightac_sdk/
|
| 227 |
+
里的 .so 都是 Linux 二进制,带上去了在客户机上只会以各种奇怪方式失败。
|
| 228 |
+
"""
|
| 229 |
+
found = []
|
| 230 |
+
for dir_path, _, files in os.walk(root):
|
| 231 |
+
for name in files:
|
| 232 |
+
full = os.path.join(dir_path, name)
|
| 233 |
+
if os.path.islink(full):
|
| 234 |
+
continue
|
| 235 |
+
try:
|
| 236 |
+
with open(full, "rb") as fh:
|
| 237 |
+
if fh.read(4) == b"\x7fELF":
|
| 238 |
+
found.append(os.path.relpath(full, root))
|
| 239 |
+
except OSError:
|
| 240 |
+
continue
|
| 241 |
+
return sorted(found)
|
| 242 |
+
|
| 243 |
+
|
| 244 |
+
def _load_paths_module():
|
| 245 |
+
"""独立加载 core/gripper/paths.py —— 故意**不经** core.gripper/__init__。
|
| 246 |
+
|
| 247 |
+
两个原因:① Windows 上 `import core.gripper` 会因顶层 `import fcntl`
|
| 248 |
+
直接失败,而打 Windows 包时本脚本仍要用它核对资源清单;
|
| 249 |
+
② __init__ 有导入期猴补丁,不必让打包进程沾上。
|
| 250 |
+
paths.py 顶层只 import os,独立加载是安全的。
|
| 251 |
+
"""
|
| 252 |
+
import importlib.util
|
| 253 |
+
path = os.path.join(ROOT, "core", "gripper", "paths.py")
|
| 254 |
+
spec = importlib.util.spec_from_file_location("_pack_lite_paths", path)
|
| 255 |
+
module = importlib.util.module_from_spec(spec)
|
| 256 |
+
spec.loader.exec_module(module)
|
| 257 |
+
return module
|
| 258 |
+
|
| 259 |
+
|
| 260 |
+
def _assert_shell_matches_paths() -> None:
|
| 261 |
+
"""start_lite.sh 的 [错误 B] 资源清单必须与 paths.required_resources() 一致。
|
| 262 |
+
|
| 263 |
+
部署脚本那时还没装 venv、不能 import,只能把 7 条路径抄成 shell 字面量;
|
| 264 |
+
这个断言把「改了 paths.py 却忘了改脚本」(→ 客户机上缺资源却照常启动、
|
| 265 |
+
夹爪静默不可见)变成**打包当场失败**。
|
| 266 |
+
"""
|
| 267 |
+
shell_path = _dist_shell_source("start_lite.sh")
|
| 268 |
+
if shell_path is None:
|
| 269 |
+
print("[警告] 找不到 start_lite.sh,跳过 [错误 B] 清单核对")
|
| 270 |
+
return
|
| 271 |
+
text = open(shell_path, encoding="utf-8").read()
|
| 272 |
+
# 脚本里路径写的是变量($GRIPPER_NATIVE 等,为的是认 KSQ_* 环境变量);
|
| 273 |
+
# 未设环境变量时它们就取默认值,先把默认值代回去再比字面量
|
| 274 |
+
normalized = (text
|
| 275 |
+
.replace("$GRIPPER_NATIVE", "core/gripper/native")
|
| 276 |
+
.replace("$GRIPPER_SIGHTAC", "core/gripper/sightac_sdk"))
|
| 277 |
+
required = _load_paths_module().required_resources()
|
| 278 |
+
missing = [os.path.relpath(p, ROOT).replace(os.sep, "/")
|
| 279 |
+
for p, _ in required.values()
|
| 280 |
+
if os.path.relpath(p, ROOT).replace(os.sep, "/") not in normalized]
|
| 281 |
+
n_checks = text.count(" check_res ")
|
| 282 |
+
if missing or n_checks != len(required):
|
| 283 |
+
print(f"[失败] {os.path.basename(shell_path)} 的 [错误 B] 清单与 "
|
| 284 |
+
f"core/gripper/paths.py:required_resources() 不一致:")
|
| 285 |
+
for rel in missing:
|
| 286 |
+
print(f" 脚本里没有: {rel}")
|
| 287 |
+
print(f" check_res 条数 {n_checks} ≠ 资源项数 {len(required)}")
|
| 288 |
+
sys.exit(1)
|
| 289 |
+
print(f"[核对] start_lite.sh 的 [错误 B] 清单与 paths.py 一致({n_checks} 项)")
|
| 290 |
+
|
| 291 |
+
|
| 292 |
+
def _verify_payload(out_dir: str, target: str) -> None:
|
| 293 |
+
"""载荷完整性硬断言。
|
| 294 |
+
|
| 295 |
+
Linux 包:软链没被展开(R3)+ 7 项必需资源逐项就位。
|
| 296 |
+
Windows 包:代码照收、平台绑定件一个不留(含「零 ELF」这条硬不变量)——
|
| 297 |
+
于是「Windows 包」在语义上就是「Linux 包去掉载荷」,而这种形态在
|
| 298 |
+
Linux 开发机上可以直接造出来验证(mv 走 native/ 即可),不必真去
|
| 299 |
+
Windows 上撞。
|
| 300 |
+
"""
|
| 301 |
+
_assert_shell_matches_paths()
|
| 302 |
+
gr = os.path.join(out_dir, GRIPPER_REL)
|
| 303 |
+
if target != "linux":
|
| 304 |
+
for name in _PLATFORM_BOUND_NAMES:
|
| 305 |
+
if os.path.exists(os.path.join(gr, name)):
|
| 306 |
+
print(f"[失败] Windows 包不该含 {GRIPPER_REL}/{name}/")
|
| 307 |
+
sys.exit(1)
|
| 308 |
+
elf = _find_elf(gr)
|
| 309 |
+
if elf:
|
| 310 |
+
print(f"[失败] Windows 包含 ELF 文件 {len(elf)} 个:{elf[:5]}")
|
| 311 |
+
sys.exit(1)
|
| 312 |
+
absent = [f"core/gripper/{f}"
|
| 313 |
+
for f in ("__init__.py", "bridge.py", "paths.py",
|
| 314 |
+
"fays_runtime.py", "fays_single.py")
|
| 315 |
+
if not os.path.isfile(os.path.join(gr, f))]
|
| 316 |
+
if not os.path.isfile(os.path.join(out_dir, "core",
|
| 317 |
+
"gripper_codec.py")):
|
| 318 |
+
absent.append("core/gripper_codec.py")
|
| 319 |
+
if not os.path.isfile(os.path.join(out_dir, "core",
|
| 320 |
+
"s80m_manager.py")):
|
| 321 |
+
absent.append("core/s80m_manager.py")
|
| 322 |
+
if absent:
|
| 323 |
+
print(f"[失败] Windows 包缺夹爪代码(两个包都收 .py): {absent}")
|
| 324 |
+
sys.exit(1)
|
| 325 |
+
print(f"[载荷] Windows 包:夹爪代码 {_count_py(gr)} 个 .py、零 ELF、"
|
| 326 |
+
f"无 {' / '.join(_PLATFORM_BOUND_NAMES)} —— "
|
| 327 |
+
f"那边的夹爪组框在、里面空着,属预期降级")
|
| 328 |
+
return
|
| 329 |
+
src = os.path.join(ROOT, GRIPPER_PAYLOAD_REL)
|
| 330 |
+
dst = os.path.join(out_dir, GRIPPER_PAYLOAD_REL)
|
| 331 |
+
if not os.path.isdir(dst):
|
| 332 |
+
print(f"[失败] Linux 包缺夹爪载荷 {GRIPPER_PAYLOAD_REL}/")
|
| 333 |
+
sys.exit(1)
|
| 334 |
+
n_src, n_dst = _count_symlinks(src), _count_symlinks(dst)
|
| 335 |
+
if n_src != n_dst:
|
| 336 |
+
print(f"[失败] 载荷软链被展开:源 {n_src} 个 → 包内 {n_dst} 个"
|
| 337 |
+
f"(copytree 必须传 symlinks=True)")
|
| 338 |
+
sys.exit(1)
|
| 339 |
+
size_mb = _tree_size(dst) / 1e6
|
| 340 |
+
size_src_mb = _tree_size(src) / 1e6
|
| 341 |
+
if size_mb > size_src_mb + 20:
|
| 342 |
+
print(f"[失败] 载荷体积异常:源 {size_src_mb:.0f}MB → 包内 {size_mb:.0f}MB")
|
| 343 |
+
sys.exit(1)
|
| 344 |
+
env = dict(os.environ, PYTHONPATH=out_dir, QT_QPA_PLATFORM="offscreen")
|
| 345 |
+
probe = ("import core.gripper.bridge, core.s80m_manager\n"
|
| 346 |
+
"from core.gripper import paths\n"
|
| 347 |
+
"missing = [k for k, (p, x) in paths.required_resources().items()\n"
|
| 348 |
+
" if not os.path.exists(p)\n"
|
| 349 |
+
" or (x and not os.access(p, os.X_OK))]\n"
|
| 350 |
+
"assert not missing, missing\n"
|
| 351 |
+
"assert paths.gripper_resources_available()\n"
|
| 352 |
+
"print('OK')")
|
| 353 |
+
proc = subprocess.run(
|
| 354 |
+
[sys.executable, "-c", "import os\n" + probe], cwd=out_dir, env=env,
|
| 355 |
+
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
|
| 356 |
+
if proc.returncode != 0:
|
| 357 |
+
print("[失败] 包内夹爪资源自检未通过:")
|
| 358 |
+
print(proc.stderr.decode("utf-8", "replace"))
|
| 359 |
+
sys.exit(1)
|
| 360 |
+
print(f"[载荷] {GRIPPER_PAYLOAD_REL}/ {size_mb:.0f}MB、"
|
| 361 |
+
f"软链 {n_dst} 个原样保留、7 项必需资源全部可执行")
|
| 362 |
+
|
| 363 |
+
|
| 364 |
+
def collect(out_dir: str, with_wheels: bool, force: bool,
|
| 365 |
+
target: str) -> None:
|
| 366 |
out_dir = os.path.abspath(out_dir)
|
| 367 |
if force:
|
| 368 |
+
# 只重建代码路径;venv_lite/wheels/data 与分发壳原样保留;
|
| 369 |
+
# core/gripper/native(470MB)也不动 —— 它不由本脚本生成,
|
| 370 |
+
# 白删白拷一轮纯浪费(Windows 包目录里本来就没有它)
|
| 371 |
+
keep = os.path.join(out_dir, GRIPPER_PAYLOAD_REL)
|
| 372 |
for rel in (CODE_FILES + ["config", "core", "ui", "tools"]):
|
| 373 |
+
_force_remove(os.path.join(out_dir, rel), keep)
|
|
|
|
|
|
|
|
|
|
|
|
|
| 374 |
elif os.path.isfile(os.path.join(out_dir, "main_lite.py")):
|
| 375 |
print(f"[跳过] 目标目录已存在: {out_dir}")
|
| 376 |
+
print(" --force 只重建代码部分,venv_lite/wheels/data 与载荷会保留")
|
| 377 |
sys.exit(0)
|
| 378 |
|
| 379 |
copied = []
|
| 380 |
+
for rel in CODE_FILES + SELFTEST_FILES + (
|
| 381 |
+
TOOL_FILES_LINUX if target == "linux" else []):
|
| 382 |
src = os.path.join(ROOT, rel)
|
| 383 |
if not os.path.isfile(src):
|
| 384 |
print(f"[警告] 缺文件: {rel}")
|
|
|
|
| 389 |
copied.append(rel)
|
| 390 |
|
| 391 |
for rel in DIST_FILES:
|
| 392 |
+
src = _dist_shell_source(rel)
|
| 393 |
+
if src is None:
|
| 394 |
+
print(f"[警告] 缺分发文件: {rel}(应在仓库根或 lite_package/ 下)")
|
| 395 |
+
continue
|
| 396 |
dst = os.path.join(out_dir, rel)
|
| 397 |
if os.path.abspath(src) == os.path.abspath(dst):
|
| 398 |
+
copied.append(rel) # 源���目标(在 lite_package/ 内收集)
|
|
|
|
|
|
|
|
|
|
| 399 |
continue
|
| 400 |
shutil.copy2(src, dst)
|
| 401 |
copied.append(rel)
|
|
|
|
| 403 |
for rel in WHOLE_DIRS:
|
| 404 |
src = os.path.join(ROOT, rel)
|
| 405 |
dst = os.path.join(out_dir, rel)
|
| 406 |
+
if rel == GRIPPER_REL:
|
| 407 |
+
ignore = _gripper_ignore_factory(
|
| 408 |
+
src, windows=(target != "linux"),
|
| 409 |
+
# 目标包已有载荷(--force 重打包):native/ 整棵跳过,
|
| 410 |
+
# 只刷新 Python 代码 —— 也避开已存在软链的 File exists
|
| 411 |
+
keep_existing_native=os.path.isdir(
|
| 412 |
+
os.path.join(dst, "native")))
|
| 413 |
+
else:
|
| 414 |
+
ignore = shutil.ignore_patterns("__pycache__")
|
| 415 |
+
# symlinks=True 是硬要求(载荷里 76 个相对软链,展开 +330MB)
|
| 416 |
+
shutil.copytree(src, dst, symlinks=True, ignore=ignore,
|
| 417 |
+
dirs_exist_ok=True)
|
| 418 |
copied.append(rel + "/")
|
| 419 |
|
| 420 |
for sub, files in MODULE_FILES.items():
|
|
|
|
| 448 |
print("[失败] 目标目录内 import main_lite 自检未通过:")
|
| 449 |
print(proc.stderr.decode("utf-8", "replace"))
|
| 450 |
sys.exit(1)
|
| 451 |
+
_verify_payload(out_dir, target)
|
| 452 |
|
| 453 |
total = sum(os.path.getsize(os.path.join(out_dir, r))
|
| 454 |
for r in copied if os.path.isfile(os.path.join(out_dir, r)))
|
| 455 |
+
if target == "linux":
|
| 456 |
+
payload = _tree_size(os.path.join(out_dir, GRIPPER_REL)) / 1e6
|
| 457 |
+
hint = (f" 客户机落盘 = 本目录 + 夹爪载荷 {payload:.0f}MB"
|
| 458 |
+
f" + venv_lite(自动安装, ~750MB) ≈ {payload / 1000 + 0.75:.2f}GB"
|
| 459 |
+
+ (" + wheels/" if with_wheels else ""))
|
| 460 |
+
else:
|
| 461 |
+
hint = (" 客户机落盘 = 本目录 + venv_lite(自动安装, ~750MB) ≈ 0.75GB"
|
| 462 |
+
"(Windows 包不含夹爪载荷 native/ 与 sightac_sdk/,"
|
| 463 |
+
"那边的夹爪组框在、里面空着)"
|
| 464 |
+
+ (" + wheels/" if with_wheels else ""))
|
| 465 |
print(f"[完成] 收集 {len(copied)} 个文件到 {out_dir}/,"
|
| 466 |
+
f"代码共 {total / 1024:.0f} KB,import main_lite 自检通过({target})")
|
| 467 |
+
print(hint)
|
|
|
|
| 468 |
|
| 469 |
|
| 470 |
def main() -> None:
|
| 471 |
parser = argparse.ArgumentParser(description="收集极简采集版发布包")
|
| 472 |
parser.add_argument("--out", default=os.path.join(ROOT, "lite_package"),
|
| 473 |
help="输出目录(默认仓库根 lite_package/)")
|
| 474 |
+
parser.add_argument("--target", default=None,
|
| 475 |
+
choices=("linux", "windows"),
|
| 476 |
+
help="目标平台(默认本机)。windows 包不含平台绑定件"
|
| 477 |
+
"(native/sightac_sdk),夹爪代码两个包同源")
|
| 478 |
parser.add_argument("--with-wheels", action="store_true",
|
| 479 |
help="连同 wheels/ 离线依赖包一起收集")
|
| 480 |
parser.add_argument("--force", action="store_true",
|
| 481 |
+
help="重建代码部分(保留 venv_lite/wheels/data/载荷)")
|
| 482 |
args = parser.parse_args()
|
| 483 |
+
target = args.target or ("windows" if os.name == "nt" else "linux")
|
| 484 |
+
collect(os.path.abspath(args.out), args.with_wheels, args.force, target)
|
| 485 |
|
| 486 |
|
| 487 |
if __name__ == "__main__":
|
collector/{lite_package/start_lite.bat → start_lite.bat}
RENAMED
|
@@ -16,6 +16,15 @@ rem ֻ
|
|
| 16 |
rem �������� venv/ ����Ӱ�죩��wheels/ �� data/ ���汾���á�
|
| 17 |
rem ������װ˳��: wheels\ ���߰� �� �����ƾ��� �� �廪���� �� �ٷ�Դ
|
| 18 |
rem ������ A-G ��Ӧ ʹ��˵��_lite.md���쳣�����½ڣ�
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
rem ============================================================
|
| 20 |
setlocal enabledelayedexpansion
|
| 21 |
cd /d "%~dp0"
|
|
@@ -113,7 +122,7 @@ echo [5/6]
|
|
| 113 |
echo.
|
| 114 |
echo ������ָ����
|
| 115 |
echo �� �豸: �����Լ 2 ���Զ��������б���ѡ�к�� ����
|
| 116 |
-
echo �� ¼��: �ȿ� D435���ٵ� ��ʼ/ֹͣ������ֹͣ=���棬X ����=���ϣ�
|
| 117 |
echo �� �ϴ�: ¼����ɺ��Զ��ϴ��������·��б��ֶ��ϴ�
|
| 118 |
echo �� ˵��: ˫�� start_lite.bat help �� ʹ��˵��_lite.md
|
| 119 |
echo.
|
|
|
|
| 16 |
rem �������� venv/ ����Ӱ�죩��wheels/ �� data/ ���汾���á�
|
| 17 |
rem ������װ˳��: wheels\ ���߰� �� �����ƾ��� �� �廪���� �� �ٷ�Դ
|
| 18 |
rem ������ A-G ��Ӧ ʹ��˵��_lite.md���쳣�����½ڣ�
|
| 19 |
+
rem
|
| 20 |
+
rem ����צΪʲô���ﲻ�顿Windows ��**���ⲻ��** UMI/Fays ��צ��ԭ����Դ
|
| 21 |
+
rem ��core/gripper/native��Լ 460MB���봥�� SDK��core/gripper/sightac_sdk��
|
| 22 |
+
rem ������ pyarmor ����ʱ�� .so �� libSonixCamera.so������ ��Щȫ�� ELF��
|
| 23 |
+
rem �� Windows �������ܲ��ˡ���צ�� Python ������������������ƽ̨����
|
| 24 |
+
rem ȱ�غ�ʱ core/gripper/paths.py �� gripper_resources_available() ���ؼ�
|
| 25 |
+
rem �� �豸�б���ġ�UMI ��צ������ڣ��������ǿյģ��㡸��������ʾ���б���û���豸��������**Ԥ�ڽ���**�����ǹ��ϡ�
|
| 26 |
+
rem Linux �����غɣ��DZ� start_lite.sh �� [���� B] ����У�飻���߿���
|
| 27 |
+
rem ���Գƣ��������ﲹһ������Դȱʧ���ļ�飨����ݰ�������Զ��ͨ������
|
| 28 |
rem ============================================================
|
| 29 |
setlocal enabledelayedexpansion
|
| 30 |
cd /d "%~dp0"
|
|
|
|
| 122 |
echo.
|
| 123 |
echo ������ָ����
|
| 124 |
echo �� �豸: �����Լ 2 ���Զ��������б���ѡ�к�� ����
|
| 125 |
+
echo �� ¼��: �ȿ� D435 / UVC ����ͷ / ��צ������һ����ƵԴ�����ٵ� ��ʼ/ֹͣ������ֹͣ=���棬X ����=���ϣ�
|
| 126 |
echo �� �ϴ�: ¼����ɺ��Զ��ϴ��������·��б��ֶ��ϴ�
|
| 127 |
echo �� ˵��: ˫�� start_lite.bat help �� ʹ��˵��_lite.md
|
| 128 |
echo.
|
collector/{lite_package/start_lite.sh → start_lite.sh}
RENAMED
|
@@ -10,6 +10,9 @@
|
|
| 10 |
# 只安装 requirements-lite.txt 白名单依赖(独立 venv_lite/,
|
| 11 |
# 与主程序 venv/ 互不影响);wheels/ 与 data/ 两版本共用。
|
| 12 |
# 依赖安装顺序: 离线 wheels/ 包 → 阿里云镜像 → 清华镜像 → 官方源
|
|
|
|
|
|
|
|
|
|
| 13 |
# ============================================================
|
| 14 |
set -euo pipefail
|
| 15 |
cd "$(dirname "$0")"
|
|
@@ -32,6 +35,48 @@ if [ ! -f main_lite.py ] || [ ! -f requirements-lite.txt ]; then
|
|
| 32 |
exit 1
|
| 33 |
fi
|
| 34 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 35 |
# ── [1/6] 定位 Python(>= 3.10,推荐 3.12)──
|
| 36 |
PY=""
|
| 37 |
find_python() {
|
|
@@ -111,7 +156,7 @@ echo "[5/6] 启动极简采集 ..."
|
|
| 111 |
echo
|
| 112 |
echo "【操作指引】"
|
| 113 |
echo " · 设备: 插入后约 2 秒自动出现在列表,选中后点 开启"
|
| 114 |
-
echo " · 录制: 先开 D435,再点 开始/停止(正常停止=保存,⛔ 丢弃=作废)"
|
| 115 |
echo " · 上传: 录制完成后自动上传,或在下方列表手动上传"
|
| 116 |
echo " · 说明: ./start_lite.sh help 打开 使用说明_lite.md"
|
| 117 |
echo
|
|
|
|
| 10 |
# 只安装 requirements-lite.txt 白名单依赖(独立 venv_lite/,
|
| 11 |
# 与主程序 venv/ 互不影响);wheels/ 与 data/ 两版本共用。
|
| 12 |
# 依赖安装顺序: 离线 wheels/ 包 → 阿里云镜像 → 清华镜像 → 官方源
|
| 13 |
+
#
|
| 14 |
+
# 夹爪(UMI/Fays S80M): 原生资源 core/gripper/native(约 460MB)随本包
|
| 15 |
+
# 下发,[B] 会逐项校验,缺失报 [错误 B] 并拒启;Windows 包不带该资源。
|
| 16 |
# ============================================================
|
| 17 |
set -euo pipefail
|
| 18 |
cd "$(dirname "$0")"
|
|
|
|
| 35 |
exit 1
|
| 36 |
fi
|
| 37 |
|
| 38 |
+
# ── [B] 夹爪原生资源自检(Linux 包独有,缺失即拒启)──
|
| 39 |
+
# 夹爪(UMI / Fays S80M)的原生栈约 460MB,作为**必需资源**随 Linux 分发包
|
| 40 |
+
# 下发;Windows 包不带(原生栈是 ELF x86-64,物理上跑不了,那边靠载荷缺失
|
| 41 |
+
# 自然降级为纯手套版,见 start_lite.bat 的注释)。
|
| 42 |
+
# 清单与 core/gripper/paths.py 的 required_resources() 一一对应:这里只查
|
| 43 |
+
# 存在性与可执行位(不 import —— 那要等 venv 装好,见 [E]);打包脚本
|
| 44 |
+
# scripts/pack_lite.py 会在收集时逐项核对这份清单,改了 paths.py 忘了改这里
|
| 45 |
+
# 会**在打包当场报错**,所以两处不会悄悄漂移。
|
| 46 |
+
# 逃生舱 KSQ_GRIPPER_SKIP_CHECK=1:决策上载荷是必需资源,但硬停启动的代价
|
| 47 |
+
# 是整机不可用,留一个显式开关(跳过则夹爪组为空,D435/UVC/手套照常)。
|
| 48 |
+
GRIPPER_NATIVE="${KSQ_GRIPPER_NATIVE_ROOT:-core/gripper/native}"
|
| 49 |
+
GRIPPER_SIGHTAC="${KSQ_SIGHTAC_ROOT:-core/gripper/sightac_sdk}"
|
| 50 |
+
if [ "${KSQ_GRIPPER_SKIP_CHECK:-0}" = "1" ]; then
|
| 51 |
+
echo "[B] KSQ_GRIPPER_SKIP_CHECK=1 —— 跳过夹爪资源自检,夹爪将不可用"
|
| 52 |
+
else
|
| 53 |
+
MISSING=""
|
| 54 |
+
check_res() { # $1=标签 $2=路径 $3=1 表示需要可执行位
|
| 55 |
+
if [ ! -e "$2" ]; then
|
| 56 |
+
MISSING="${MISSING} · $1 —— 缺 $2\n"
|
| 57 |
+
elif [ "$3" = "1" ] && [ ! -x "$2" ]; then
|
| 58 |
+
MISSING="${MISSING} · $1 —— $2 没有可执行位(chmod +x 修复)\n"
|
| 59 |
+
fi
|
| 60 |
+
}
|
| 61 |
+
check_res "Fays ORB 桥接" "$GRIPPER_NATIVE/dist/fays_opencv48/bin/fayssense_orb_slam_sn219_opencv48_mark_only" 1
|
| 62 |
+
check_res "Fays 标定探测" "$GRIPPER_NATIVE/dist/fays_aikit/bin/fays_vikit_calibration_probe" 1
|
| 63 |
+
check_res "ORB-SLAM3 核心库" "$GRIPPER_NATIVE/dist/orb_mark_only/lib/libORB_SLAM3.so" 0
|
| 64 |
+
check_res "ORB 词典" "$GRIPPER_NATIVE/ORBvoc.txt" 0
|
| 65 |
+
check_res "libuvc 相机服务" "$GRIPPER_NATIVE/camera_service/build/ksq-camera-service" 1
|
| 66 |
+
check_res "Sightac SDK" "$GRIPPER_SIGHTAC/api_new" 0
|
| 67 |
+
check_res "Fays SDK lib" "$GRIPPER_NATIVE/FaysSense_VI_Kit_Release/lib" 0
|
| 68 |
+
if [ -n "$MISSING" ]; then
|
| 69 |
+
echo "[错误 B] 夹爪原生资源缺失 —— 本分发包不完整:"
|
| 70 |
+
printf '%b' "$MISSING"
|
| 71 |
+
echo " 原因通常是解压少了一层目录、拷贝中断,或覆盖升级时漏了 core/gripper/native/。"
|
| 72 |
+
echo " 修复: ① 重新完整解压分发包;② 从原始包补齐 core/gripper/native/(约 460MB);"
|
| 73 |
+
echo " ③ 仅想先用 D435/UVC/手套: KSQ_GRIPPER_SKIP_CHECK=1 ./start_lite.sh"
|
| 74 |
+
echo " 自定义安装位置时可用 KSQ_GRIPPER_NATIVE_ROOT=<native 目录> 指路。"
|
| 75 |
+
exit 1
|
| 76 |
+
fi
|
| 77 |
+
echo "[B] 夹爪资源自检通过(7 项)"
|
| 78 |
+
fi
|
| 79 |
+
|
| 80 |
# ── [1/6] 定位 Python(>= 3.10,推荐 3.12)──
|
| 81 |
PY=""
|
| 82 |
find_python() {
|
|
|
|
| 156 |
echo
|
| 157 |
echo "【操作指引】"
|
| 158 |
echo " · 设备: 插入后约 2 秒自动出现在列表,选中后点 开启"
|
| 159 |
+
echo " · 录制: 先开 D435 / UVC 摄像头 / 夹爪(至少一个视频源),再点 开始/停止(正常停止=保存,⛔ 丢弃=作废)"
|
| 160 |
echo " · 上传: 录制完成后自动上传,或在下方列表手动上传"
|
| 161 |
echo " · 说明: ./start_lite.sh help 打开 使用说明_lite.md"
|
| 162 |
echo
|
collector/tools/dedup_gripper_native.py
ADDED
|
@@ -0,0 +1,205 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""回收 core/gripper/native 里被展开成实体文件的软链副本(只动布局,不动内容)。
|
| 2 |
+
|
| 3 |
+
背景:厂商交付的 kit 在传输途中走过 ZIP/Windows 或 `cp -rL`,标准 `.so`
|
| 4 |
+
布局被解引用成实体副本。实测整棵树有 339 MB 是同一内容的重复存放,最夸张
|
| 5 |
+
的是 orb48_env 里同一个 libopenblas 存了 7 份(7 × 39.6 MB,md5 全一致),
|
| 6 |
+
而正常布局是「一个 libopenblasp-r0.3.34.so + 6 个软链」。
|
| 7 |
+
|
| 8 |
+
判定规则(保守,三条同时满足才动):
|
| 9 |
+
1. 同一个目录内(跨目录的重复一律不碰——不同目录可能是各自独立的变体,
|
| 10 |
+
例如 dist/orb_mark_only/lib 与 lib_system_boost 各有一份 libORB_SLAM3.so)
|
| 11 |
+
2. 内容逐字节相同(md5)
|
| 12 |
+
3. 文件名是共享库名(`.so` / `.so.3` / `.so.4.2.0` 这种)
|
| 13 |
+
|
| 14 |
+
保留哪一份:名字最长的那个(`libopencv_core.so.4.2.0` 胜过 `.so.4.2` 与
|
| 15 |
+
`.so`;`libopenblasp-r0.3.34.so` 胜过 `libblas.so.3`)——这与 conda/系统
|
| 16 |
+
打包时「实体文件用最长版本名、其余做软链」的实际布局一致。字节既然相同,
|
| 17 |
+
即便选错也只是软链朝向问题,不影响任何加载行为。
|
| 18 |
+
|
| 19 |
+
不动的(只列出来给人看):
|
| 20 |
+
- 备份文件(`.pre_*_backup` 这类名字,不匹配共享库名)
|
| 21 |
+
- 跨目录重复
|
| 22 |
+
- 可执行文件(bin/ 下的产物没有 .so 后缀)
|
| 23 |
+
|
| 24 |
+
软链一律用**相对路径**,这样整棵树 `cp -a` 搬走后依然解析。
|
| 25 |
+
|
| 26 |
+
每次 --apply 都会落一份清单(默认 logs/gripper_native_dedup_<时间戳>.json),
|
| 27 |
+
记录每个被替换文件的路径、md5、体积与软链目标,凭它可以原样还原:
|
| 28 |
+
ln -sf 的反操作 = 删软链 + 从 target 复制回来。
|
| 29 |
+
|
| 30 |
+
用法:
|
| 31 |
+
python tools/dedup_gripper_native.py # dry-run,只打印清单
|
| 32 |
+
python tools/dedup_gripper_native.py --apply # 实际执行
|
| 33 |
+
python tools/dedup_gripper_native.py --root <目录> # 指定其它树
|
| 34 |
+
"""
|
| 35 |
+
from __future__ import annotations
|
| 36 |
+
|
| 37 |
+
import argparse
|
| 38 |
+
import collections
|
| 39 |
+
import hashlib
|
| 40 |
+
import json
|
| 41 |
+
import os
|
| 42 |
+
import re
|
| 43 |
+
import sys
|
| 44 |
+
import time
|
| 45 |
+
|
| 46 |
+
ROOT_DEFAULT = os.path.join(
|
| 47 |
+
os.path.dirname(os.path.dirname(os.path.abspath(__file__))),
|
| 48 |
+
"core", "gripper", "native",
|
| 49 |
+
)
|
| 50 |
+
|
| 51 |
+
# 共享库名:xxx.so / xxx.so.3 / xxx.so.4.2.0(可带 -r0.3.34 这类内部版本)
|
| 52 |
+
_SHARED_LIB_RE = re.compile(r"^.+\.so(\.[0-9]+)*$")
|
| 53 |
+
|
| 54 |
+
MIN_SIZE = 300 * 1024 # 小于 300KB 的不值得动(软链本身也要占 inode)
|
| 55 |
+
|
| 56 |
+
|
| 57 |
+
def _md5(path: str) -> str:
|
| 58 |
+
h = hashlib.md5()
|
| 59 |
+
with open(path, "rb") as fh:
|
| 60 |
+
for chunk in iter(lambda: fh.read(1 << 20), b""):
|
| 61 |
+
h.update(chunk)
|
| 62 |
+
return h.hexdigest()
|
| 63 |
+
|
| 64 |
+
|
| 65 |
+
def scan(root: str):
|
| 66 |
+
"""返回 (可回收组, 需人工判断的重复组)。"""
|
| 67 |
+
by_key = collections.defaultdict(list)
|
| 68 |
+
for dirpath, dirnames, filenames in os.walk(root):
|
| 69 |
+
dirnames[:] = [d for d in dirnames if d != "__pycache__"]
|
| 70 |
+
for name in filenames:
|
| 71 |
+
path = os.path.join(dirpath, name)
|
| 72 |
+
try:
|
| 73 |
+
if os.path.islink(path) or os.path.getsize(path) < MIN_SIZE:
|
| 74 |
+
continue
|
| 75 |
+
by_key[(dirpath, _md5(path))].append((name, path))
|
| 76 |
+
except OSError:
|
| 77 |
+
continue
|
| 78 |
+
|
| 79 |
+
reclaimable, review = [], []
|
| 80 |
+
for (dirpath, digest), items in by_key.items():
|
| 81 |
+
if len(items) < 2:
|
| 82 |
+
continue
|
| 83 |
+
size = os.path.getsize(items[0][1])
|
| 84 |
+
group = {
|
| 85 |
+
"dir": dirpath,
|
| 86 |
+
"md5": digest,
|
| 87 |
+
"size": size,
|
| 88 |
+
"names": sorted(n for n, _ in items),
|
| 89 |
+
"paths": sorted(p for _, p in items),
|
| 90 |
+
}
|
| 91 |
+
if all(_SHARED_LIB_RE.match(n) for n, _ in items):
|
| 92 |
+
reclaimable.append(group)
|
| 93 |
+
else:
|
| 94 |
+
review.append(group)
|
| 95 |
+
|
| 96 |
+
# 跨目录的重复单独提出来(不进可回收组)
|
| 97 |
+
by_digest = collections.defaultdict(list)
|
| 98 |
+
for (dirpath, digest), items in by_key.items():
|
| 99 |
+
for name, path in items:
|
| 100 |
+
by_digest[digest].append(path)
|
| 101 |
+
cross = []
|
| 102 |
+
for digest, paths in by_digest.items():
|
| 103 |
+
dirs = {os.path.dirname(p) for p in paths}
|
| 104 |
+
if len(dirs) > 1:
|
| 105 |
+
cross.append({
|
| 106 |
+
"md5": digest,
|
| 107 |
+
"size": os.path.getsize(paths[0]),
|
| 108 |
+
"paths": sorted(paths),
|
| 109 |
+
})
|
| 110 |
+
cross.sort(key=lambda g: -g["size"])
|
| 111 |
+
return reclaimable, review, cross
|
| 112 |
+
|
| 113 |
+
|
| 114 |
+
def plan_for(group):
|
| 115 |
+
"""选最长名字那份当实体,其余转软链(相对路径)。"""
|
| 116 |
+
paths = sorted(group["paths"], key=lambda p: (-len(os.path.basename(p)),
|
| 117 |
+
os.path.basename(p)))
|
| 118 |
+
target = paths[0]
|
| 119 |
+
return target, paths[1:]
|
| 120 |
+
|
| 121 |
+
|
| 122 |
+
def main() -> None:
|
| 123 |
+
ap = argparse.ArgumentParser(description=__doc__,
|
| 124 |
+
formatter_class=argparse.RawDescriptionHelpFormatter)
|
| 125 |
+
ap.add_argument("--root", default=ROOT_DEFAULT)
|
| 126 |
+
ap.add_argument("--apply", action="store_true", help="实际执行��默认 dry-run)")
|
| 127 |
+
ap.add_argument("--manifest", default="",
|
| 128 |
+
help="清单输出路径(默认 logs/gripper_native_dedup_<时间戳>.json)")
|
| 129 |
+
args = ap.parse_args()
|
| 130 |
+
|
| 131 |
+
root = os.path.abspath(args.root)
|
| 132 |
+
if not os.path.isdir(root):
|
| 133 |
+
sys.exit(f"目录不存在: {root}")
|
| 134 |
+
|
| 135 |
+
reclaimable, review, cross = scan(root)
|
| 136 |
+
|
| 137 |
+
total = sum(g["size"] * (len(g["paths"]) - 1) for g in reclaimable)
|
| 138 |
+
print(f"扫描: {root}")
|
| 139 |
+
print(f"可回收组: {len(reclaimable)} 组,合计 {total / 1048576:.0f} MB\n")
|
| 140 |
+
|
| 141 |
+
entries = []
|
| 142 |
+
for group in sorted(reclaimable, key=lambda g: -g["size"] * len(g["paths"])):
|
| 143 |
+
target, aliases = plan_for(group)
|
| 144 |
+
rel_target = os.path.relpath(target, group["dir"])
|
| 145 |
+
print(f" {group['size'] * len(aliases) / 1048576:6.1f} MB "
|
| 146 |
+
f"{os.path.relpath(group['dir'], root)}/")
|
| 147 |
+
print(f" 实体 → {os.path.basename(target)}")
|
| 148 |
+
for alias in aliases:
|
| 149 |
+
print(f" 软链 ← {os.path.basename(alias)}")
|
| 150 |
+
entries.append({
|
| 151 |
+
"path": alias, "action": "symlink",
|
| 152 |
+
"target": rel_target, "size": group["size"],
|
| 153 |
+
"md5": group["md5"],
|
| 154 |
+
})
|
| 155 |
+
print()
|
| 156 |
+
|
| 157 |
+
if review:
|
| 158 |
+
wasted = sum(g["size"] * (len(g["paths"]) - 1) for g in review)
|
| 159 |
+
print(f"需人工判断(名字非共享库名,未动): {len(review)} 组 / {wasted / 1048576:.0f} MB")
|
| 160 |
+
for group in sorted(review, key=lambda g: -g["size"])[:10]:
|
| 161 |
+
print(f" {group['size'] / 1048576:5.1f} MB × {len(group['paths'])} "
|
| 162 |
+
f"— {', '.join(os.path.basename(n) for n in group['names'][:4])}")
|
| 163 |
+
print()
|
| 164 |
+
|
| 165 |
+
if cross:
|
| 166 |
+
wasted = sum(g["size"] * (len(g["paths"]) - 1) for g in cross)
|
| 167 |
+
print(f"跨目录重复(可能是各自独立的变体,未动): {len(cross)} 组 / {wasted / 1048576:.0f} MB")
|
| 168 |
+
for group in cross[:6]:
|
| 169 |
+
print(f" {group['size'] / 1048576:5.1f} MB —")
|
| 170 |
+
for p in group["paths"]:
|
| 171 |
+
print(f" {os.path.relpath(p, root)}")
|
| 172 |
+
print()
|
| 173 |
+
|
| 174 |
+
if not args.apply:
|
| 175 |
+
print("dry-run 结束,未做任何修改。加 --apply 执行。")
|
| 176 |
+
return
|
| 177 |
+
|
| 178 |
+
manifest = args.manifest or os.path.join(
|
| 179 |
+
os.path.dirname(os.path.dirname(os.path.abspath(__file__))),
|
| 180 |
+
"logs", f"gripper_native_dedup_{time.strftime('%Y%m%d_%H%M%S')}.json")
|
| 181 |
+
os.makedirs(os.path.dirname(manifest), exist_ok=True)
|
| 182 |
+
|
| 183 |
+
done = 0
|
| 184 |
+
for entry in entries:
|
| 185 |
+
alias = entry["path"]
|
| 186 |
+
link_dir = os.path.dirname(alias)
|
| 187 |
+
os.remove(alias) # 删副本(内容在 target 里还在)
|
| 188 |
+
os.symlink(entry["target"], alias) # 相对软链
|
| 189 |
+
done += 1
|
| 190 |
+
assert os.path.realpath(alias) == os.path.realpath(
|
| 191 |
+
os.path.join(link_dir, entry["target"])), alias
|
| 192 |
+
|
| 193 |
+
with open(manifest, "w", encoding="utf-8") as fh:
|
| 194 |
+
json.dump({"root": root, "applied_at": time.time(),
|
| 195 |
+
"reclaimed_bytes": total, "entries": entries},
|
| 196 |
+
fh, ensure_ascii=False, indent=2)
|
| 197 |
+
|
| 198 |
+
saved = sum(os.path.getsize(p) for p in [] ) or total
|
| 199 |
+
print(f"[完成] {done} 个副本转成软链,回收 {total / 1048576:.0f} MB")
|
| 200 |
+
print(f" 清单: {manifest}")
|
| 201 |
+
print(" 还原: 删软链 + 从 target 复制回原文件(清单里有 md5 可校验)")
|
| 202 |
+
|
| 203 |
+
|
| 204 |
+
if __name__ == "__main__":
|
| 205 |
+
main()
|
collector/tools/tests/lite_gripper_smoke_test.py
ADDED
|
@@ -0,0 +1,415 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""极简版 UMI 夹爪冒烟测试(无硬件,离屏 Qt,三条路径)。
|
| 2 |
+
|
| 3 |
+
路径 D(UI + 假桥注入):
|
| 4 |
+
LiteWindow(gripper_bridge_cls=FakeGripperBridge) → 只开夹爪(无 D435/UVC)
|
| 5 |
+
→ 夹爪 RGB 作主视频源 → 录制 3s → 停 → 校验 gripper_rgb 视频 /
|
| 6 |
+
parquet 的力/力矩阵/夹爪状态/SLAM 轨迹列 / info.json 的 devices 与
|
| 7 |
+
features.scale。顺带断言「只录不显」契约(预览区没有被夹爪污染)。
|
| 8 |
+
|
| 9 |
+
路径 E(纯管线,无 UI):
|
| 10 |
+
rig2 前缀(gripper_2_*)的完整落盘链:力 / 力矩阵 / 状态 / 轨迹 +
|
| 11 |
+
set_force_matrix_spec → 校验双夹爪命名空间与倍率登记。
|
| 12 |
+
|
| 13 |
+
路径 F(编码器搬家回归):
|
| 14 |
+
core.gripper_codec.encode_gripper_force_matrix 必须**就是**
|
| 15 |
+
ui.main_window 里那个函数对象(re-export 而非复制第二份)。
|
| 16 |
+
venv_lite 装不下 ui.main_window 的重依赖,那里跳过。
|
| 17 |
+
|
| 18 |
+
用法:
|
| 19 |
+
QT_QPA_PLATFORM=offscreen venv/bin/python tools/tests/lite_gripper_smoke_test.py
|
| 20 |
+
(包内副本用 lite_package/venv_lite/bin/python 跑,验证闭包完整性)
|
| 21 |
+
"""
|
| 22 |
+
import json
|
| 23 |
+
import os
|
| 24 |
+
import shutil
|
| 25 |
+
import sys
|
| 26 |
+
import threading
|
| 27 |
+
import time
|
| 28 |
+
|
| 29 |
+
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.dirname(
|
| 30 |
+
os.path.abspath(__file__)))))
|
| 31 |
+
os.environ.setdefault("QT_QPA_PLATFORM", "offscreen")
|
| 32 |
+
|
| 33 |
+
import numpy as np
|
| 34 |
+
import pyarrow.parquet as pq
|
| 35 |
+
from PyQt5.QtCore import QObject, QTimer, pyqtSignal
|
| 36 |
+
from PyQt5.QtWidgets import QApplication, QMessageBox
|
| 37 |
+
|
| 38 |
+
from config import settings
|
| 39 |
+
from core.device_detector import DeviceInfo
|
| 40 |
+
from core.gripper_codec import encode_gripper_force_matrix
|
| 41 |
+
from core.helpers import (episode_video_files, pooled_data_parquet_path,
|
| 42 |
+
pooled_info_path)
|
| 43 |
+
from core.pipeline import CameraPipeline
|
| 44 |
+
from ui.lite_window import LiteUploadManager, LiteWindow
|
| 45 |
+
|
| 46 |
+
OUT_ROOT_D = "/tmp/lite_gripper_smoke_ui"
|
| 47 |
+
OUT_ROOT_E = "/tmp/lite_gripper_smoke_pipe"
|
| 48 |
+
DURATION_D = 3.0
|
| 49 |
+
DURATION_E = 3.0
|
| 50 |
+
|
| 51 |
+
# 力矩阵列名(rig1 无前缀;rig2 加 "gripper_2_")
|
| 52 |
+
MATRIX_LEN = settings.GRIPPER_FORCE_MATRIX_DIM ** 2 * 3
|
| 53 |
+
|
| 54 |
+
|
| 55 |
+
def wait(app, ms):
|
| 56 |
+
t_end = time.time() + ms / 1000.0
|
| 57 |
+
while time.time() < t_end:
|
| 58 |
+
app.processEvents()
|
| 59 |
+
time.sleep(0.01)
|
| 60 |
+
|
| 61 |
+
|
| 62 |
+
class FakeGripperBridge(QObject):
|
| 63 |
+
"""GripperBridge 替身(鸭子类型,信号口径与 core/gripper/bridge.py 一致)。
|
| 64 |
+
|
| 65 |
+
★ 刻意**不提供 stereo_frame_ready** —— lite 只录不显,不接双目信号;
|
| 66 |
+
哪天有人给 lite 接上它,这里会直接 AttributeError 暴露。
|
| 67 |
+
力矩阵走与真桥相同的 latest-wins 单槽 + 序号口径(泵线程按 seq 判新)。
|
| 68 |
+
"""
|
| 69 |
+
|
| 70 |
+
rgb_frame_ready = pyqtSignal(object, object)
|
| 71 |
+
tactile_ready = pyqtSignal(str, object, object, object, object)
|
| 72 |
+
pose_ready = pyqtSignal(object, object, object, object, object)
|
| 73 |
+
gripper_state_ready = pyqtSignal(object)
|
| 74 |
+
opened = pyqtSignal()
|
| 75 |
+
log = pyqtSignal(str)
|
| 76 |
+
error = pyqtSignal(str)
|
| 77 |
+
closed = pyqtSignal()
|
| 78 |
+
|
| 79 |
+
TICK_MS = 33
|
| 80 |
+
MATRIX_EVERY = 5 # 力矩阵降频:30Hz 全量(每帧 187500 个数)太占地
|
| 81 |
+
|
| 82 |
+
def __init__(self, parent=None, stereo_slot="gripper_stereo_left",
|
| 83 |
+
rig_index=1):
|
| 84 |
+
super().__init__(parent)
|
| 85 |
+
self.stereo_slot = stereo_slot
|
| 86 |
+
self.rig_index = int(rig_index)
|
| 87 |
+
self.serial = ""
|
| 88 |
+
self.closed_count = 0
|
| 89 |
+
self._lock = threading.Lock()
|
| 90 |
+
self._seq = {"left": 0, "right": 0}
|
| 91 |
+
self._matrix = {} # side → (capture_ns, ndarray)
|
| 92 |
+
self._tick = 0
|
| 93 |
+
self._dropped = 0
|
| 94 |
+
self._elapsed = 0
|
| 95 |
+
self._timer = QTimer(self)
|
| 96 |
+
self._timer.timeout.connect(self._emit)
|
| 97 |
+
|
| 98 |
+
# ── 生命周期(UI 调)──
|
| 99 |
+
|
| 100 |
+
def open(self, esp_serial):
|
| 101 |
+
self.serial = esp_serial
|
| 102 |
+
self._timer.start(self.TICK_MS)
|
| 103 |
+
self.opened.emit()
|
| 104 |
+
|
| 105 |
+
def close(self):
|
| 106 |
+
self._timer.stop()
|
| 107 |
+
self.closed_count += 1
|
| 108 |
+
self.closed.emit()
|
| 109 |
+
|
| 110 |
+
# ── 力矩阵泵线程读的口(真桥为加锁单槽 latest-wins)──
|
| 111 |
+
|
| 112 |
+
def matrix_seq(self, side):
|
| 113 |
+
with self._lock:
|
| 114 |
+
return self._seq.get(side, 0)
|
| 115 |
+
|
| 116 |
+
def latest_matrix_stamped(self, side):
|
| 117 |
+
with self._lock:
|
| 118 |
+
entry = self._matrix.get(side)
|
| 119 |
+
return None if entry is None else (int(entry[0]), entry[1])
|
| 120 |
+
|
| 121 |
+
def stereo_drop_snapshot(self):
|
| 122 |
+
return (self._dropped, self._elapsed)
|
| 123 |
+
|
| 124 |
+
def reset_stereo_drop_watch(self):
|
| 125 |
+
self._dropped = 0
|
| 126 |
+
self._elapsed = 0
|
| 127 |
+
|
| 128 |
+
def clear_trajectory(self):
|
| 129 |
+
pass
|
| 130 |
+
|
| 131 |
+
# ── 样本生产(模拟 30Hz 的相机/触觉/串口三条流)──
|
| 132 |
+
|
| 133 |
+
@staticmethod
|
| 134 |
+
def make_matrix(seed):
|
| 135 |
+
m = np.zeros((settings.GRIPPER_FORCE_MATRIX_DIM,
|
| 136 |
+
settings.GRIPPER_FORCE_MATRIX_DIM, 3), dtype=np.float32)
|
| 137 |
+
i = 100 + (seed % 20)
|
| 138 |
+
m[i:i + 10, i:i + 10, 0] = 5.0 + (seed % 7)
|
| 139 |
+
return m
|
| 140 |
+
|
| 141 |
+
def _emit(self):
|
| 142 |
+
self._tick += 1
|
| 143 |
+
now_ns = time.monotonic_ns()
|
| 144 |
+
self._elapsed += 1
|
| 145 |
+
if self._tick % 9 == 0: # 模拟偶发双目空桶(走汇总日志分支)
|
| 146 |
+
self._dropped += 1
|
| 147 |
+
frame = np.zeros((480, 640, 3), dtype=np.uint8)
|
| 148 |
+
frame[..., 2] = (self._tick * 5) & 255
|
| 149 |
+
self.rgb_frame_ready.emit(frame, now_ns)
|
| 150 |
+
for side in ("left", "right"):
|
| 151 |
+
force = [float(100 + self._tick), 2.0, 3.0]
|
| 152 |
+
matrix = None
|
| 153 |
+
if self._tick % self.MATRIX_EVERY == 0:
|
| 154 |
+
matrix = self.make_matrix(self._tick)
|
| 155 |
+
with self._lock:
|
| 156 |
+
self._seq[side] += 1
|
| 157 |
+
self._matrix[side] = (now_ns, matrix)
|
| 158 |
+
self.tactile_ready.emit(
|
| 159 |
+
side, np.zeros((240, 320, 3), dtype=np.uint8), force,
|
| 160 |
+
matrix, now_ns)
|
| 161 |
+
pos = (0.001 * self._tick, 0.002 * self._tick, 0.0)
|
| 162 |
+
self.pose_ready.emit(pos, (0.0, 0.0, 0.0, 1.0), (pos,),
|
| 163 |
+
float(self._tick) * 0.033,
|
| 164 |
+
287607700000000 + self._tick * 33333333)
|
| 165 |
+
self.gripper_state_ready.emit(
|
| 166 |
+
{"pct": 40.0 + self._tick % 10, "gripped": True, "fz": 1200.0})
|
| 167 |
+
|
| 168 |
+
|
| 169 |
+
# ── 路径 D: UI + 假夹爪桥(只开夹爪,无 D435/UVC)──
|
| 170 |
+
|
| 171 |
+
def path_d():
|
| 172 |
+
shutil.rmtree(OUT_ROOT_D, ignore_errors=True)
|
| 173 |
+
app = QApplication(sys.argv)
|
| 174 |
+
settings.UPLOAD_AUTO_SYNC = False
|
| 175 |
+
settings.ENCODER_PROBE_ENABLED = False # lite 口径:直接 x264,测试提速
|
| 176 |
+
|
| 177 |
+
pipe = CameraPipeline(output_dir=OUT_ROOT_D)
|
| 178 |
+
finished = []
|
| 179 |
+
pipe.recording_aborted.connect(lambda sid: finished.append("ABORTED"))
|
| 180 |
+
pipe.recording_finished.connect(lambda sid, path: finished.append(path))
|
| 181 |
+
win = LiteWindow(
|
| 182 |
+
pipeline=pipe,
|
| 183 |
+
upload_manager=LiteUploadManager("http://127.0.0.1:9"),
|
| 184 |
+
gripper_bridge_cls=FakeGripperBridge)
|
| 185 |
+
# 守卫若拦下会弹模态框 —— 离屏下没人点确定,测试会永久挂住;
|
| 186 |
+
# 换成收集器,让「被拦」表现为可读的断言失败
|
| 187 |
+
guard_msgs = []
|
| 188 |
+
QMessageBox.warning = staticmethod(
|
| 189 |
+
lambda *a, **k: guard_msgs.append(a[2] if len(a) > 2 else ""))
|
| 190 |
+
win.show()
|
| 191 |
+
wait(app, 200)
|
| 192 |
+
|
| 193 |
+
dev = DeviceInfo(key="gripper:FAKE01", kind="gripper",
|
| 194 |
+
display_name="UMI 夹爪", serial="FAKE01",
|
| 195 |
+
address="/dev/ttyACM0")
|
| 196 |
+
assert win._open_gripper(dev), "假夹爪开启失败"
|
| 197 |
+
bridge = win._workers[dev.key]["bridge"]
|
| 198 |
+
assert isinstance(bridge, FakeGripperBridge)
|
| 199 |
+
assert bridge.serial == "FAKE01", f"open() 未拿到序列号: {bridge.serial}"
|
| 200 |
+
wait(app, 800)
|
| 201 |
+
|
| 202 |
+
entry = win._workers[dev.key]
|
| 203 |
+
assert entry["slot_map"]["rgb"] in entry["_video_registered"], \
|
| 204 |
+
"夹爪 RGB 槽未惰性注册(首帧没到?)"
|
| 205 |
+
# 只录不显契约:预览区/预览源完全没有夹爪
|
| 206 |
+
assert win._src_combo.count() == 3, f"预览源被改动: {win._src_combo.count()}"
|
| 207 |
+
assert not [s for s in win._ring if "gripper" in s], \
|
| 208 |
+
f"预览 ring 收到夹爪帧: {list(win._ring)}"
|
| 209 |
+
assert "pose_view" not in entry and "side_widgets" not in entry, \
|
| 210 |
+
"lite 条目不该有显示控件"
|
| 211 |
+
meta = win._device_meta() # 显式分支:不许 KeyError
|
| 212 |
+
gmeta = [m for m in meta if m["kind"] == "gripper"]
|
| 213 |
+
assert len(gmeta) == 1 and len(gmeta[0]["slots"]) == 5, f"meta 异常: {meta}"
|
| 214 |
+
print(f" D: 夹爪已开启({bridge.stereo_slot} / rig{bridge.rig_index}),"
|
| 215 |
+
f"RGB 槽 {entry['slot_map']['rgb']} 已注册,预览区未受影响")
|
| 216 |
+
|
| 217 |
+
# 只开夹爪时录制守卫必须放行(夹爪 RGB 作主视频源)
|
| 218 |
+
assert win._master_video_slot() == "gripper_rgb", \
|
| 219 |
+
f"主视频源选择错误: {win._master_video_slot()}"
|
| 220 |
+
win._task_edit.setText("lite_gripper")
|
| 221 |
+
win._start_recording()
|
| 222 |
+
assert not guard_msgs, f"录制守卫拦下了只开夹爪的录制: {guard_msgs}"
|
| 223 |
+
assert pipe.is_recording, "只开夹爪时录制未启动(守卫拦住了?)"
|
| 224 |
+
wait(app, int(DURATION_D * 1000))
|
| 225 |
+
win._stop_recording()
|
| 226 |
+
t_end = time.time() + 15
|
| 227 |
+
while not finished and time.time() < t_end:
|
| 228 |
+
wait(app, 50)
|
| 229 |
+
assert finished and finished[0] != "ABORTED", f"录制异常: {finished}"
|
| 230 |
+
task_dir = finished[0]
|
| 231 |
+
ep = pipe.last_episode_index
|
| 232 |
+
assert ep > 0, "episode 序号异常"
|
| 233 |
+
assert win._workers[dev.key]["matrix_pump_stop"] is None, \
|
| 234 |
+
"录制结束未停力矩阵泵"
|
| 235 |
+
win.close()
|
| 236 |
+
wait(app, 200)
|
| 237 |
+
assert bridge.closed_count == 1, "关窗未关闭桥接(进程会残留)"
|
| 238 |
+
assert dev.key not in win._workers, "关窗未回收夹爪条目"
|
| 239 |
+
|
| 240 |
+
videos = episode_video_files(task_dir, ep)
|
| 241 |
+
assert "gripper_rgb" in videos, f"夹爪 RGB 视频缺失: {videos}"
|
| 242 |
+
assert os.path.getsize(videos["gripper_rgb"]) > 1000, "夹爪视频过小"
|
| 243 |
+
tbl = pq.read_table(pooled_data_parquet_path(task_dir, ep))
|
| 244 |
+
assert tbl.num_rows > 0, "parquet 无行"
|
| 245 |
+
cols = tbl.column_names
|
| 246 |
+
want = ["observation.gripper_state",
|
| 247 |
+
"observation.gripper_left_force", "observation.gripper_right_force",
|
| 248 |
+
"observation.gripper_left_force_matrix",
|
| 249 |
+
"observation.gripper_right_force_matrix",
|
| 250 |
+
"observation.gripper_left_force_ns",
|
| 251 |
+
"observation.gripper_left_force_matrix_ns",
|
| 252 |
+
"observation.slam_trajectory",
|
| 253 |
+
# 逐点宿主时刻:这条覆盖的是完整 lite UI 接线(假桥 pose_ready
|
| 254 |
+
# 第 5 参 → lite_window lambda → _on_gripper_pose → pipeline),
|
| 255 |
+
# 信号参数个数或 lambda 形参写错这里就会缺列
|
| 256 |
+
"observation.slam_trajectory_ns"]
|
| 257 |
+
for name in want:
|
| 258 |
+
assert name in cols, f"缺列 {name}(有: {cols})"
|
| 259 |
+
# 轨迹点列与其时刻列必须逐行等长(下标即配对)
|
| 260 |
+
_traj = tbl.column("observation.slam_trajectory").to_pylist()
|
| 261 |
+
_ns = tbl.column("observation.slam_trajectory_ns").to_pylist()
|
| 262 |
+
assert any(_ns), "slam_trajectory_ns 全空(假桥发了戳却没落盘)"
|
| 263 |
+
assert all(len(a) // 8 == len(b) for a, b in zip(_traj, _ns)), \
|
| 264 |
+
"slam_trajectory 与 slam_trajectory_ns 长度不同步"
|
| 265 |
+
mat = tbl.column("observation.gripper_left_force_matrix").to_pylist()
|
| 266 |
+
rows = [r for r in mat if r]
|
| 267 |
+
assert rows and len(rows[0]) == MATRIX_LEN, \
|
| 268 |
+
f"力矩阵长度异常: {len(rows[0]) if rows else 0} != {MATRIX_LEN}"
|
| 269 |
+
traj = [r for r in tbl.column("observation.slam_trajectory").to_pylist() if r]
|
| 270 |
+
assert traj and len(traj[0]) % 8 == 0, "轨迹列不是 8 值/点"
|
| 271 |
+
firsts = [row[0] for row in traj]
|
| 272 |
+
assert firsts == sorted(firsts), "轨迹时间戳非单调"
|
| 273 |
+
info = json.load(open(pooled_info_path(task_dir), encoding="utf-8"))
|
| 274 |
+
kinds = [d.get("kind") for d in info.get("devices", [])]
|
| 275 |
+
assert "gripper" in kinds, f"info.json devices 缺 gripper: {kinds}"
|
| 276 |
+
spec = settings.GRIPPER_FORCE_MATRIX_SPEC
|
| 277 |
+
scale = settings.GRIPPER_FORCE_MATRIX_SCALES.get(spec)
|
| 278 |
+
feat = info["features"]["observation.gripper_left_force_matrix"]
|
| 279 |
+
assert feat["scale"] == (scale or 1), \
|
| 280 |
+
f"features.scale 与规格不符: {feat} vs spec={spec}"
|
| 281 |
+
print(f" D: {DURATION_D}s → gripper_rgb {os.path.getsize(videos['gripper_rgb']) // 1024}KB"
|
| 282 |
+
f" / {tbl.num_rows} 行 / 力矩阵 {len(rows)} 行 x{MATRIX_LEN}"
|
| 283 |
+
f" / 轨迹 {len(traj)} 行 / scale={feat['scale']}")
|
| 284 |
+
print(" D: PASS")
|
| 285 |
+
return 0
|
| 286 |
+
|
| 287 |
+
|
| 288 |
+
# ── 路径 E: 纯管线 + rig2 命名空间(无 UI、无桥)──
|
| 289 |
+
|
| 290 |
+
def path_e():
|
| 291 |
+
shutil.rmtree(OUT_ROOT_E, ignore_errors=True)
|
| 292 |
+
app = QApplication.instance() or QApplication(sys.argv)
|
| 293 |
+
settings.UPLOAD_AUTO_SYNC = False
|
| 294 |
+
settings.ENCODER_PROBE_ENABLED = False
|
| 295 |
+
|
| 296 |
+
prefix = "gripper_2_" # 第二台夹爪的数据列命名空间
|
| 297 |
+
spec = settings.GRIPPER_FORCE_MATRIX_SPEC
|
| 298 |
+
pipe = CameraPipeline(output_dir=OUT_ROOT_E)
|
| 299 |
+
finished = []
|
| 300 |
+
pipe.recording_finished.connect(lambda sid, path: finished.append(path))
|
| 301 |
+
# 倍率只能挂在 recording_started 之后登记:start_recording 立刻返回,
|
| 302 |
+
# writer 由后台线程创建(pipeline._start_async)且在 start_episode 里
|
| 303 |
+
# 会清空已登记的倍率 —— 在 start_recording 之后紧跟着调是**静默无效**的
|
| 304 |
+
pipe.recording_started.connect(
|
| 305 |
+
lambda sid: pipe.set_force_matrix_spec(prefix, spec))
|
| 306 |
+
pipe.register_external_source(f"{prefix}rgb", (480, 640), fps=30.0)
|
| 307 |
+
pipe.start_recording(f"{prefix}rgb", task_name="lite_gripper2",
|
| 308 |
+
device_meta=[{"key": "gripper:FAKE02",
|
| 309 |
+
"kind": "gripper", "name": "UMI 夹爪",
|
| 310 |
+
"serial": "FAKE02",
|
| 311 |
+
"slots": [f"{prefix}rgb"]}])
|
| 312 |
+
|
| 313 |
+
rgb = np.zeros((480, 640, 3), dtype=np.uint8)
|
| 314 |
+
matrix = FakeGripperBridge.make_matrix(7)
|
| 315 |
+
t0 = time.time()
|
| 316 |
+
state = {"n": 0}
|
| 317 |
+
|
| 318 |
+
def feed():
|
| 319 |
+
state["n"] += 1
|
| 320 |
+
rgb[..., 1] = (state["n"] * 3) & 255
|
| 321 |
+
pipe.write_external_frame(f"{prefix}rgb", rgb.copy(), hardware_ns=0)
|
| 322 |
+
if pipe._writer is None:
|
| 323 |
+
return
|
| 324 |
+
ns = time.monotonic_ns()
|
| 325 |
+
pipe.write_tactile_force("left", [1.5, 2.5, 3.5], prefix=prefix,
|
| 326 |
+
capture_ns=ns)
|
| 327 |
+
pipe.write_tactile_force("right", [4.5, 5.5, 6.5], prefix=prefix,
|
| 328 |
+
capture_ns=ns)
|
| 329 |
+
if state["n"] % 5 == 0:
|
| 330 |
+
pipe.write_tactile_force_matrix(
|
| 331 |
+
"left", encode_gripper_force_matrix(matrix, spec),
|
| 332 |
+
prefix=prefix, capture_ns=ns)
|
| 333 |
+
pipe.write_tactile_force_matrix(
|
| 334 |
+
"right", encode_gripper_force_matrix(matrix, spec),
|
| 335 |
+
prefix=prefix, capture_ns=ns)
|
| 336 |
+
pipe.write_gripper_state([55.0, 1.0, 900.0], prefix=prefix)
|
| 337 |
+
pipe.write_slam_trajectory([0.1, 0.2, 0.3, 0.0, 0.0, 0.0, 1.0,
|
| 338 |
+
float(state["n"]) * 0.03], prefix=prefix,
|
| 339 |
+
timestamp=float(state["n"]) * 0.03,
|
| 340 |
+
host_ns=time.monotonic_ns())
|
| 341 |
+
if time.time() - t0 >= DURATION_E:
|
| 342 |
+
feed_timer.stop()
|
| 343 |
+
pipe.finish_recording("")
|
| 344 |
+
|
| 345 |
+
feed_timer = QTimer()
|
| 346 |
+
feed_timer.timeout.connect(feed)
|
| 347 |
+
feed_timer.start(30)
|
| 348 |
+
|
| 349 |
+
t_end = time.time() + 20
|
| 350 |
+
while not finished and time.time() < t_end:
|
| 351 |
+
app.processEvents()
|
| 352 |
+
time.sleep(0.01)
|
| 353 |
+
assert finished, "未收到 recording_finished"
|
| 354 |
+
task_dir = finished[0]
|
| 355 |
+
ep = pipe.last_episode_index
|
| 356 |
+
|
| 357 |
+
tbl = pq.read_table(pooled_data_parquet_path(task_dir, ep))
|
| 358 |
+
cols = tbl.column_names
|
| 359 |
+
for side in ("left", "right"):
|
| 360 |
+
for suffix in ("force", "force_ns", "force_matrix", "force_matrix_ns"):
|
| 361 |
+
name = f"observation.{prefix}gripper_{side}_{suffix}"
|
| 362 |
+
assert name in cols, f"缺 rig2 列 {name}(有: {cols})"
|
| 363 |
+
for bare in ("observation.gripper_state", "observation.slam_trajectory",
|
| 364 |
+
"observation.gripper_left_force"):
|
| 365 |
+
assert bare not in cols, f"rig2 录制不该有裸 rig1 列 {bare}"
|
| 366 |
+
# rig2 的逐点宿主时刻列:与轨迹同序等长(下标即配对)是本列的硬契约
|
| 367 |
+
ns_col = f"observation.{prefix}slam_trajectory_ns"
|
| 368 |
+
assert ns_col in cols, f"缺 rig2 时刻列 {ns_col}(有: {cols})"
|
| 369 |
+
traj2 = tbl.column(f"observation.{prefix}slam_trajectory").to_pylist()
|
| 370 |
+
ns2 = tbl.column(ns_col).to_pylist()
|
| 371 |
+
assert all(len(a) // 8 == len(b) for a, b in zip(traj2, ns2)), \
|
| 372 |
+
"rig2 轨迹点列与时刻列长度不同步"
|
| 373 |
+
assert any(ns2), "rig2 时刻列全空(写了 host_ns 却没落盘)"
|
| 374 |
+
assert all(all(v > 0 for v in b) for a, b in zip(traj2, ns2) if a), \
|
| 375 |
+
"rig2 有点的行出现了 0 时刻(与「0=未知」的约定混淆)"
|
| 376 |
+
info = json.load(open(pooled_info_path(task_dir), encoding="utf-8"))
|
| 377 |
+
feat = info["features"][
|
| 378 |
+
f"observation.{prefix}gripper_left_force_matrix"]
|
| 379 |
+
expect = settings.GRIPPER_FORCE_MATRIX_SCALES.get(spec) or 1
|
| 380 |
+
assert feat["scale"] == expect, f"rig2 倍率未登记: {feat} != {expect}"
|
| 381 |
+
print(f" E: rig2 前缀列全在({tbl.num_rows} 行),"
|
| 382 |
+
f"无 rig1 裸列,features.scale={feat['scale']}")
|
| 383 |
+
print(" E: PASS")
|
| 384 |
+
return 0
|
| 385 |
+
|
| 386 |
+
|
| 387 |
+
# ── 路径 F: 编码器搬家回归(主程序 re-export 同一函数对象)──
|
| 388 |
+
|
| 389 |
+
def path_f():
|
| 390 |
+
try:
|
| 391 |
+
import ui.main_window as mw
|
| 392 |
+
except Exception as exc:
|
| 393 |
+
print(f" F: 跳过(ui.main_window 不可导入: {type(exc).__name__}: {exc})")
|
| 394 |
+
return 0
|
| 395 |
+
assert mw.encode_gripper_force_matrix is encode_gripper_force_matrix, \
|
| 396 |
+
"ui.main_window 未从 core.gripper_codec re-export(出现第二份实现)"
|
| 397 |
+
assert (mw.describe_gripper_matrix_spec
|
| 398 |
+
is __import__("core.gripper_codec", fromlist=["x"]
|
| 399 |
+
).describe_gripper_matrix_spec), \
|
| 400 |
+
"describe_gripper_matrix_spec 未 re-export"
|
| 401 |
+
print(" F: ui.main_window 的编码器就是 core.gripper_codec 那个函数对象")
|
| 402 |
+
print(" F: PASS")
|
| 403 |
+
return 0
|
| 404 |
+
|
| 405 |
+
|
| 406 |
+
def main():
|
| 407 |
+
for fn in (path_d, path_e, path_f):
|
| 408 |
+
rc = fn()
|
| 409 |
+
if rc:
|
| 410 |
+
return rc
|
| 411 |
+
return 0
|
| 412 |
+
|
| 413 |
+
|
| 414 |
+
if __name__ == "__main__":
|
| 415 |
+
sys.exit(main())
|
collector/tools/tests/lite_import_guard_test.py
CHANGED
|
@@ -43,7 +43,10 @@ BLACKLIST = [
|
|
| 43 |
"core.task_service", "core.glove_keypoint_solver",
|
| 44 |
"core.render_engine", "core.sensor_hand_config",
|
| 45 |
"core.hand_tracking", "core.hand_processor", "core.auto_labeler",
|
| 46 |
-
|
|
|
|
|
|
|
|
|
|
| 47 |
"core.session_catalog", "core.recording_repository",
|
| 48 |
"core.task_record", "core.device_manager", "core.device_naming",
|
| 49 |
"core.exposure_controller",
|
|
@@ -58,6 +61,8 @@ WHITELIST = [
|
|
| 58 |
"core.d435_camera", "core.camera", "core.device_detector",
|
| 59 |
"core.ble_engine", "core.usb_glove_engine",
|
| 60 |
"core.uploader", "core.api_client", "core.database", "core.helpers",
|
|
|
|
|
|
|
| 61 |
]
|
| 62 |
|
| 63 |
|
|
@@ -68,6 +73,18 @@ def main():
|
|
| 68 |
win.close()
|
| 69 |
app.processEvents()
|
| 70 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 71 |
bad = sorted(m for m in BLACKLIST
|
| 72 |
if any(k == m or k.startswith(m + ".")
|
| 73 |
for k in sys.modules))
|
|
@@ -86,7 +103,7 @@ def main():
|
|
| 86 |
print("FAIL")
|
| 87 |
return 1
|
| 88 |
print(f"PASS: {len(WHITELIST)} 个白名单模块全部就位,"
|
| 89 |
-
f"禁止名单无导入,编码器探针已关闭")
|
| 90 |
return 0
|
| 91 |
|
| 92 |
|
|
|
|
| 43 |
"core.task_service", "core.glove_keypoint_solver",
|
| 44 |
"core.render_engine", "core.sensor_hand_config",
|
| 45 |
"core.hand_tracking", "core.hand_processor", "core.auto_labeler",
|
| 46 |
+
# 注:core.s80m_manager 2026-09-16 起**在 lite 闭包内**(UMI 夹爪的
|
| 47 |
+
# core/gripper/bridge.py 顶层 import 它的丢帧常量),已从禁止名单移出,
|
| 48 |
+
# 与 core/gripper/* 一起由下面的夹爪闭包段断言。
|
| 49 |
+
"core.session_loader", "core.session_timeline",
|
| 50 |
"core.session_catalog", "core.recording_repository",
|
| 51 |
"core.task_record", "core.device_manager", "core.device_naming",
|
| 52 |
"core.exposure_controller",
|
|
|
|
| 61 |
"core.d435_camera", "core.camera", "core.device_detector",
|
| 62 |
"core.ble_engine", "core.usb_glove_engine",
|
| 63 |
"core.uploader", "core.api_client", "core.database", "core.helpers",
|
| 64 |
+
# 力矩阵编码的家(ui.main_window 只是 re-export)—— lite 直接用
|
| 65 |
+
"core.gripper_codec",
|
| 66 |
]
|
| 67 |
|
| 68 |
|
|
|
|
| 73 |
win.close()
|
| 74 |
app.processEvents()
|
| 75 |
|
| 76 |
+
# 夹爪闭包:扫描线程只在**真有夹爪插着**时才走到 core.gripper.bridge,
|
| 77 |
+
# 且它在后台线程里(win.close() 早于它完成 → 断言会 flaky),所以这里
|
| 78 |
+
# 显式导入一遍,让下面的禁止名单覆盖整个夹爪闭包 —— 确认它没把
|
| 79 |
+
# torch/scipy 之类的重依赖拖进来(那是 lite 体积与客户机装不上的红线)。
|
| 80 |
+
# 只在 Linux 上有意义:原生栈是 ELF,且 core/gripper 顶层 `import fcntl`
|
| 81 |
+
# 在 Windows 上直接导入失败——那正是 Windows 包的预期降级路径。
|
| 82 |
+
gripper_state = "跳过(非 Linux)"
|
| 83 |
+
if sys.platform.startswith("linux"):
|
| 84 |
+
import core.gripper.bridge # noqa: F401
|
| 85 |
+
import core.s80m_manager # noqa: F401
|
| 86 |
+
gripper_state = "已导入并断言"
|
| 87 |
+
|
| 88 |
bad = sorted(m for m in BLACKLIST
|
| 89 |
if any(k == m or k.startswith(m + ".")
|
| 90 |
for k in sys.modules))
|
|
|
|
| 103 |
print("FAIL")
|
| 104 |
return 1
|
| 105 |
print(f"PASS: {len(WHITELIST)} 个白名单模块全部就位,"
|
| 106 |
+
f"禁止名单无导入,编码器探针已关闭,夹爪闭包 {gripper_state}")
|
| 107 |
return 0
|
| 108 |
|
| 109 |
|
collector/tools/tests/probe_gripper_rates.py
CHANGED
|
@@ -57,8 +57,11 @@ def main():
|
|
| 57 |
def on_stereo(slot, frame, hw_ns, s=s):
|
| 58 |
s["stereo_l"] += 1
|
| 59 |
|
| 60 |
-
def on_pose(pos, quat, traj=(), timestamp=None, s=s):
|
| 61 |
s["pose"] += 1
|
|
|
|
|
|
|
|
|
|
| 62 |
if s["pose_first"] is None:
|
| 63 |
s["pose_first"] = time.monotonic()
|
| 64 |
raw = b._raw_client
|
|
|
|
| 57 |
def on_stereo(slot, frame, hw_ns, s=s):
|
| 58 |
s["stereo_l"] += 1
|
| 59 |
|
| 60 |
+
def on_pose(pos, quat, traj=(), timestamp=None, host_ns=None, s=s):
|
| 61 |
s["pose"] += 1
|
| 62 |
+
# host_ns 只在签名里显式收下、本探针不消费:PyQt5 会把多余实参
|
| 63 |
+
# 静默截断给「收得少」的槽,漏写这个参数不会报错只会悄悄丢戳,
|
| 64 |
+
# 所以宁可显式列出也不靠截断兜底。
|
| 65 |
if s["pose_first"] is None:
|
| 66 |
s["pose_first"] = time.monotonic()
|
| 67 |
raw = b._raw_client
|
collector/tools/tests/smoke_gripper_p3.py
CHANGED
|
@@ -54,12 +54,18 @@ def main():
|
|
| 54 |
f"{'None' if matrix is None else matrix.shape}",
|
| 55 |
flush=True)
|
| 56 |
|
| 57 |
-
def on_pose(pos, quat, traj=(), timestamp=None):
|
| 58 |
stats["pose"] += 1
|
| 59 |
stats["last_pos"] = pos
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 60 |
if stats["pose"] <= 3 or stats["pose"] % 100 == 0:
|
| 61 |
print(f"[POSE] #{stats['pose']} pos={tuple(pos)} "
|
| 62 |
-
f"quat={tuple(quat)}
|
|
|
|
| 63 |
|
| 64 |
def on_opened():
|
| 65 |
print("[OPENED] 相机 + 触觉 + SLAM 链路就绪", flush=True)
|
|
|
|
| 54 |
f"{'None' if matrix is None else matrix.shape}",
|
| 55 |
flush=True)
|
| 56 |
|
| 57 |
+
def on_pose(pos, quat, traj=(), timestamp=None, host_ns=None):
|
| 58 |
stats["pose"] += 1
|
| 59 |
stats["last_pos"] = pos
|
| 60 |
+
# host_ns 是取样帧的宿主单调钟纳秒(与 hardware_ns 同时基);None =
|
| 61 |
+
# 当前 native 二进制未打印 Host 字段(只做了 Python 侧改动、没重编
|
| 62 |
+
# 原生库时的预期状态)。真机验收时应看到非 None、且逐点单调递增。
|
| 63 |
+
if host_ns is not None:
|
| 64 |
+
stats["host_ns_last"] = host_ns
|
| 65 |
if stats["pose"] <= 3 or stats["pose"] % 100 == 0:
|
| 66 |
print(f"[POSE] #{stats['pose']} pos={tuple(pos)} "
|
| 67 |
+
f"quat={tuple(quat)} t={timestamp} "
|
| 68 |
+
f"host_ns={host_ns}", flush=True)
|
| 69 |
|
| 70 |
def on_opened():
|
| 71 |
print("[OPENED] 相机 + 触觉 + SLAM 链路就绪", flush=True)
|
collector/tools/tests/test_gripper_bridge.py
CHANGED
|
@@ -190,8 +190,8 @@ def _run_open_flow(fail_select=None, wait_ready=True, ready_errors=(),
|
|
| 190 |
lambda s, h, f, m, ns: stats["tactile"].append(
|
| 191 |
(s, tuple(f), m, ns)))
|
| 192 |
bridge.pose_ready.connect(
|
| 193 |
-
lambda p, q, t, ts: stats["pose"].append(
|
| 194 |
-
(tuple(p), tuple(q), tuple(t), ts)))
|
| 195 |
bridge.gripper_state_ready.connect(
|
| 196 |
lambda payload: stats["state"].append(payload))
|
| 197 |
bridge.opened.connect(lambda: stats.__setitem__(
|
|
@@ -330,21 +330,34 @@ def main():
|
|
| 330 |
"paired 两包配对只发左目、右目包丢弃")
|
| 331 |
bridge2.deleteLater()
|
| 332 |
|
| 333 |
-
# SLAM 位姿投递(pos3 + quat4 + 轨迹元组 + timestamp)
|
| 334 |
pose_count = len(stats["pose"])
|
| 335 |
pose = SimpleNamespace(position=(1.0, 2.0, 3.0),
|
| 336 |
-
rotation=(0.0, 0.0, 0.0, 1.0), timestamp=0.5
|
|
|
|
| 337 |
bridge._on_slam_pose(pose)
|
| 338 |
_pump_until(lambda: len(stats["pose"]) == pose_count + 1)
|
| 339 |
check(stats["pose"][-1] == ((1.0, 2.0, 3.0), (0.0, 0.0, 0.0, 1.0),
|
| 340 |
-
((1.0, 2.0, 3.0),), 0.5
|
| 341 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 342 |
# 第二个位姿追加进轨迹(同一元组缓存对象随新点重建)
|
| 343 |
pose2 = SimpleNamespace(position=(2.0, 3.0, 4.0),
|
| 344 |
-
rotation=(0.0, 0.0, 0.0, 1.0), timestamp=0.6
|
|
|
|
| 345 |
bridge._on_slam_pose(pose2)
|
| 346 |
-
_pump_until(lambda: len(stats["pose"]) == pose_count +
|
| 347 |
-
check(stats["pose"][-1][2] == ((1.0, 2.0, 3.0), (
|
|
|
|
| 348 |
"轨迹随位姿滚动累积")
|
| 349 |
|
| 350 |
# P4 串口链接线:构造注入 on_board_update/on_grip_check,握手用 esp_tty
|
|
|
|
| 190 |
lambda s, h, f, m, ns: stats["tactile"].append(
|
| 191 |
(s, tuple(f), m, ns)))
|
| 192 |
bridge.pose_ready.connect(
|
| 193 |
+
lambda p, q, t, ts, ns: stats["pose"].append(
|
| 194 |
+
(tuple(p), tuple(q), tuple(t), ts, ns)))
|
| 195 |
bridge.gripper_state_ready.connect(
|
| 196 |
lambda payload: stats["state"].append(payload))
|
| 197 |
bridge.opened.connect(lambda: stats.__setitem__(
|
|
|
|
| 330 |
"paired 两包配对只发左目、右目包丢弃")
|
| 331 |
bridge2.deleteLater()
|
| 332 |
|
| 333 |
+
# SLAM 位姿投递(pos3 + quat4 + 轨迹元组 + timestamp + 宿主时刻)
|
| 334 |
pose_count = len(stats["pose"])
|
| 335 |
pose = SimpleNamespace(position=(1.0, 2.0, 3.0),
|
| 336 |
+
rotation=(0.0, 0.0, 0.0, 1.0), timestamp=0.5,
|
| 337 |
+
host_mono_ns=287607702508638)
|
| 338 |
bridge._on_slam_pose(pose)
|
| 339 |
_pump_until(lambda: len(stats["pose"]) == pose_count + 1)
|
| 340 |
check(stats["pose"][-1] == ((1.0, 2.0, 3.0), (0.0, 0.0, 0.0, 1.0),
|
| 341 |
+
((1.0, 2.0, 3.0),), 0.5,
|
| 342 |
+
287607702508638),
|
| 343 |
+
"pose_ready 信号携带 pos/quat/traj/timestamp/host_ns")
|
| 344 |
+
# 无 host_mono_ns 的位姿(旧 native 二进制 / 外部实现)也必须正常投递,
|
| 345 |
+
# 戳缺失只是让下游退回行号配对,不该断掉整条位姿流
|
| 346 |
+
pose_legacy = SimpleNamespace(position=(1.5, 2.5, 3.5),
|
| 347 |
+
rotation=(0.0, 0.0, 0.0, 1.0),
|
| 348 |
+
timestamp=0.55)
|
| 349 |
+
bridge._on_slam_pose(pose_legacy)
|
| 350 |
+
_pump_until(lambda: len(stats["pose"]) == pose_count + 2)
|
| 351 |
+
check(stats["pose"][-1][4] is None,
|
| 352 |
+
"缺 host_mono_ns 的位姿投递为 None(不抛异常)")
|
| 353 |
# 第二个位姿追加进轨迹(同一元组缓存对象随新点重建)
|
| 354 |
pose2 = SimpleNamespace(position=(2.0, 3.0, 4.0),
|
| 355 |
+
rotation=(0.0, 0.0, 0.0, 1.0), timestamp=0.6,
|
| 356 |
+
host_mono_ns=287607702641000)
|
| 357 |
bridge._on_slam_pose(pose2)
|
| 358 |
+
_pump_until(lambda: len(stats["pose"]) == pose_count + 3)
|
| 359 |
+
check(stats["pose"][-1][2] == ((1.0, 2.0, 3.0), (1.5, 2.5, 3.5),
|
| 360 |
+
(2.0, 3.0, 4.0)),
|
| 361 |
"轨迹随位姿滚动累积")
|
| 362 |
|
| 363 |
# P4 串口链接线:构造注入 on_board_update/on_grip_check,握手用 esp_tty
|
collector/tools/tests/test_gripper_pose_axes.py
CHANGED
|
@@ -150,7 +150,19 @@ def main():
|
|
| 150 |
atol=1e-12):
|
| 151 |
failures.append("双应用旋转应为 R_z(+180°)·R")
|
| 152 |
|
| 153 |
-
# 7.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 154 |
traj = [
|
| 155 |
PoseSample(position=tuple(float(v) for v in rng.uniform(-1, 1, 3)),
|
| 156 |
rotation=random_unit_quaternion(rng), timestamp=float(i))
|
|
|
|
| 150 |
atol=1e-12):
|
| 151 |
failures.append("双应用旋转应为 R_z(+180°)·R")
|
| 152 |
|
| 153 |
+
# 7. host_mono_ns(取样帧宿主时刻)必须原样透传:rotate_pose_z90 是
|
| 154 |
+
# 纯世界重标、不动时基。丢了它位姿就退回"只能按行号/相机钟对齐",
|
| 155 |
+
# 而落盘侧正是靠它与 hardware_ns 直接比较来配 slam 点↔视频帧。
|
| 156 |
+
stamped = PoseSample(position=(1.0, 2.0, 3.0),
|
| 157 |
+
rotation=random_unit_quaternion(rng), timestamp=1.0,
|
| 158 |
+
host_mono_ns=287607702508638)
|
| 159 |
+
if rotate_pose_z90(stamped).host_mono_ns != 287607702508638:
|
| 160 |
+
failures.append("rotate_pose_z90 丢了 host_mono_ns")
|
| 161 |
+
# 无戳(旧 native 二进制)透传 None,不能被伪造成 0
|
| 162 |
+
if rotate_pose_z90(pose).host_mono_ns is not None:
|
| 163 |
+
failures.append("无戳位姿的 host_mono_ns 应为 None")
|
| 164 |
+
|
| 165 |
+
# 8. 轨迹形状不变(等距):相邻点距离一致
|
| 166 |
traj = [
|
| 167 |
PoseSample(position=tuple(float(v) for v in rng.uniform(-1, 1, 3)),
|
| 168 |
rotation=random_unit_quaternion(rng), timestamp=float(i))
|
collector/tools/tests/test_gripper_pose_host_stamp.py
ADDED
|
@@ -0,0 +1,143 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
# -*- coding: utf-8 -*-
|
| 3 |
+
"""位姿行 Host 字段(取样帧宿主时刻)解析契约自检(离线,无硬件)。
|
| 4 |
+
|
| 5 |
+
venv/bin/python tools/tests/test_gripper_pose_host_stamp.py
|
| 6 |
+
|
| 7 |
+
背景:`slam_trajectory` 每点的 `t` 是**相机传感器钟**,而视频行的
|
| 8 |
+
`hardware_ns` 是**宿主单调钟**,两者不同源、只差一个只能拟合的偏移
|
| 9 |
+
(实测反推估计散布 ~104ms,已超一个帧间隔)。v1.3.6 起 native 在取样帧
|
| 10 |
+
进进程时取一次 CLOCK_MONOTONIC,随帧走到 stdout:
|
| 11 |
+
|
| 12 |
+
[0.0000] XYZ:(...) Quat:(...) Host:(287607702508638)
|
| 13 |
+
|
| 14 |
+
本测试钉死解析侧的向后兼容:**现场已部署的旧 native 二进制不打印 Host
|
| 15 |
+
字段**,Python 侧升级后必须照旧能解析那些行(host_mono_ns = None),
|
| 16 |
+
否则整条位姿流会断——采集链路直接不可用。
|
| 17 |
+
|
| 18 |
+
覆盖:
|
| 19 |
+
1. 新格式(带 Host)→ 全部字段正确、host_mono_ns 取到
|
| 20 |
+
2. 旧格式(无 Host)→ 照旧解析,host_mono_ns = None(向后兼容)
|
| 21 |
+
3. 带 Δq 第二行的变体:首行仍解析出位姿(正则只吃首行)
|
| 22 |
+
4. Host 值畸形/缺失不污染其他字段
|
| 23 |
+
5. 宿主戳是 int 不是 float(纳秒精度不能被浮点吃掉)
|
| 24 |
+
6. 落盘链路的两步变换(rotate_pose_z90 + dataclasses.replace)
|
| 25 |
+
都不丢戳、不改戳
|
| 26 |
+
退出码 0 = 全部通过。
|
| 27 |
+
"""
|
| 28 |
+
|
| 29 |
+
from __future__ import annotations
|
| 30 |
+
|
| 31 |
+
import dataclasses
|
| 32 |
+
import os
|
| 33 |
+
import sys
|
| 34 |
+
|
| 35 |
+
REPO_ROOT = os.path.dirname(os.path.dirname(os.path.dirname(
|
| 36 |
+
os.path.abspath(__file__))))
|
| 37 |
+
sys.path.insert(0, REPO_ROOT)
|
| 38 |
+
|
| 39 |
+
from core.gripper.slam.protocol import ( # noqa: E402
|
| 40 |
+
_parse_pose, quat_conjugate, quat_product, rotate_pose_z90,
|
| 41 |
+
)
|
| 42 |
+
|
| 43 |
+
# 与 core/gripper/native/.../fayssense_orb_slam.cc 的 printf 逐字对应
|
| 44 |
+
NEW_LINE = ("[1.2345] XYZ:(0.1000,-0.2000,0.3000) "
|
| 45 |
+
"Quat:(w=1.0000,x=0.0000,y=0.0000,z=0.0000) "
|
| 46 |
+
"Host:(287607702508638)")
|
| 47 |
+
OLD_LINE = ("[1.2345] XYZ:(0.1000,-0.2000,0.3000) "
|
| 48 |
+
"Quat:(w=1.0000,x=0.0000,y=0.0000,z=0.0000)")
|
| 49 |
+
DQ_LINE = ("[2.0000] XYZ:(1.0000,2.0000,3.0000) "
|
| 50 |
+
"Quat:(w=0.7071,x=0.7071,y=0.0000,z=0.0000) "
|
| 51 |
+
"Host:(287607705000000)\n"
|
| 52 |
+
" Δq(10f):1.23° axis=(0.000,0.000,1.000)")
|
| 53 |
+
|
| 54 |
+
_FAILS: list[str] = []
|
| 55 |
+
|
| 56 |
+
|
| 57 |
+
def check(name: str, cond: bool, detail: str = ""):
|
| 58 |
+
tag = "PASS" if cond else "FAIL"
|
| 59 |
+
print(f" [{tag}] {name}" + (f" — {detail}" if detail else ""))
|
| 60 |
+
if not cond:
|
| 61 |
+
_FAILS.append(name)
|
| 62 |
+
|
| 63 |
+
|
| 64 |
+
def main() -> int:
|
| 65 |
+
print("[1] 新格式(native 已重编,带 Host 字段)")
|
| 66 |
+
pose = _parse_pose(NEW_LINE)
|
| 67 |
+
check("解析成功", pose is not None)
|
| 68 |
+
check("position 正确", pose.position == (0.1, -0.2, 0.3),
|
| 69 |
+
str(pose.position))
|
| 70 |
+
check("rotation 正确(w 在前 → 内部 x,y,z,w)",
|
| 71 |
+
pose.rotation == (0.0, 0.0, 0.0, 1.0), str(pose.rotation))
|
| 72 |
+
check("timestamp 正确", abs(pose.timestamp - 1.2345) < 1e-12,
|
| 73 |
+
str(pose.timestamp))
|
| 74 |
+
check("host_mono_ns = 287607702508638",
|
| 75 |
+
pose.host_mono_ns == 287607702508638, str(pose.host_mono_ns))
|
| 76 |
+
check("宿主戳是 int 不是 float(纳秒精度不被浮点吃掉)",
|
| 77 |
+
isinstance(pose.host_mono_ns, int)
|
| 78 |
+
and not isinstance(pose.host_mono_ns, bool),
|
| 79 |
+
type(pose.host_mono_ns).__name__)
|
| 80 |
+
check("valid 通过", pose.valid)
|
| 81 |
+
|
| 82 |
+
print("[2] 旧格式(现场已部署的 native 二进制,不打印 Host)")
|
| 83 |
+
old = _parse_pose(OLD_LINE)
|
| 84 |
+
check("照旧解析成功(否则整条位姿流会断)", old is not None)
|
| 85 |
+
check("几何字段与新版一致", old.position == pose.position
|
| 86 |
+
and old.rotation == pose.rotation
|
| 87 |
+
and abs(old.timestamp - pose.timestamp) < 1e-12)
|
| 88 |
+
check("host_mono_ns = None(不是 0)",
|
| 89 |
+
old.host_mono_ns is None, repr(old.host_mono_ns))
|
| 90 |
+
check("valid 通过(无戳不影响位姿有效性)", old.valid)
|
| 91 |
+
|
| 92 |
+
print("[3] Δq 变体:正则只吃首行,第二行不干扰")
|
| 93 |
+
dq = _parse_pose(DQ_LINE)
|
| 94 |
+
check("解析成功", dq is not None)
|
| 95 |
+
check("取到首行位姿", dq.position == (1.0, 2.0, 3.0), str(dq.position))
|
| 96 |
+
check("取到首行宿主戳", dq.host_mono_ns == 287607705000000,
|
| 97 |
+
str(dq.host_mono_ns))
|
| 98 |
+
|
| 99 |
+
print("[4] Host 畸形/截断不污染其他字段")
|
| 100 |
+
for bad, why in (
|
| 101 |
+
("[1.0] XYZ:(1.0,2.0,3.0) Quat:(w=1.0,x=0.0,y=0.0,z=0.0) Host:()",
|
| 102 |
+
"空括号"),
|
| 103 |
+
("[1.0] XYZ:(1.0,2.0,3.0) Quat:(w=1.0,x=0.0,y=0.0,z=0.0) Host:(abc)",
|
| 104 |
+
"非数字"),
|
| 105 |
+
("[1.0] XYZ:(1.0,2.0,3.0) Quat:(w=1.0,x=0.0,y=0.0,z=0.0) Host:(12",
|
| 106 |
+
"括号未闭合"),
|
| 107 |
+
("[1.0] XYZ:(1.0,2.0,3.0) Quat:(w=1.0,x=0.0,y=0.0,z=0.0) Host:123",
|
| 108 |
+
"缺括号"),
|
| 109 |
+
):
|
| 110 |
+
got = _parse_pose(bad)
|
| 111 |
+
check(f"{why}:位姿字段仍完好、戳退化为 None",
|
| 112 |
+
got is not None and got.position == (1.0, 2.0, 3.0)
|
| 113 |
+
and got.host_mono_ns is None,
|
| 114 |
+
f"pose={None if got is None else got.position} "
|
| 115 |
+
f"ns={None if got is None else got.host_mono_ns}")
|
| 116 |
+
|
| 117 |
+
print("[5] 落盘链路两步变换不丢戳/不改戳")
|
| 118 |
+
# _publish_pose 实际做的是:rotate_pose_z90 → dataclasses.replace
|
| 119 |
+
# (原点后姿态相对化)。两步都必须原样带过宿主戳。
|
| 120 |
+
rolled = rotate_pose_z90(pose)
|
| 121 |
+
check("rotate_pose_z90 透传", rolled.host_mono_ns == 287607702508638,
|
| 122 |
+
str(rolled.host_mono_ns))
|
| 123 |
+
zero = (0.7071051, 0.0, 0.0, -0.7071051)
|
| 124 |
+
relativized = dataclasses.replace(
|
| 125 |
+
rolled,
|
| 126 |
+
rotation=quat_product(quat_conjugate(zero), rolled.rotation))
|
| 127 |
+
check("dataclasses.replace 相对化后仍在",
|
| 128 |
+
relativized.host_mono_ns == 287607702508638,
|
| 129 |
+
str(relativized.host_mono_ns))
|
| 130 |
+
check("位置/时间戳未被这两步改动",
|
| 131 |
+
relativized.position == rolled.position
|
| 132 |
+
and relativized.timestamp == rolled.timestamp)
|
| 133 |
+
|
| 134 |
+
print()
|
| 135 |
+
if _FAILS:
|
| 136 |
+
print(f"FAILED: {len(_FAILS)} 项 — {_FAILS}")
|
| 137 |
+
return 1
|
| 138 |
+
print("ALL PASS")
|
| 139 |
+
return 0
|
| 140 |
+
|
| 141 |
+
|
| 142 |
+
if __name__ == "__main__":
|
| 143 |
+
sys.exit(main())
|
collector/tools/tests/test_gripper_slam_columns.py
CHANGED
|
@@ -17,6 +17,12 @@ slam_trajectory(变长、每点带真实时间戳)信息严格更多,故
|
|
| 17 |
3. 双 rig 前缀命名空间互不干扰
|
| 18 |
4. 删分支后 state / force / force_matrix 仍各按原类型落盘(没被 else 误吞)
|
| 19 |
5. 全空 episode(一帧夹爪数据都没有)不建任何夹爪列,且不抛异常
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 20 |
退出码 0 = 全部通过。
|
| 21 |
"""
|
| 22 |
|
|
@@ -193,6 +199,78 @@ def main() -> int:
|
|
| 193 |
check("features 无任何 slam 键",
|
| 194 |
not [k for k in read_features(os.path.join(tmp, "none"))
|
| 195 |
if "slam_" in k])
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 196 |
finally:
|
| 197 |
shutil.rmtree(tmp, ignore_errors=True)
|
| 198 |
|
|
|
|
| 17 |
3. 双 rig 前缀命名空间互不干扰
|
| 18 |
4. 删分支后 state / force / force_matrix 仍各按原类型落盘(没被 else 误吞)
|
| 19 |
5. 全空 episode(一帧夹爪数据都没有)不建任何夹爪列,且不抛异常
|
| 20 |
+
6. slam_trajectory_ns → 类型必须是 list<int64> 而非标量 int64(钉死
|
| 21 |
+
「专用分支排在通用 _ns 之前」)、与 slam_trajectory 逐行等长、
|
| 22 |
+
features dtype/shape/encoding
|
| 23 |
+
7. pipeline 侧 write_slam_trajectory/_pop_gripper_snapshots:时刻与点
|
| 24 |
+
锁步、缺戳补 0 不跳过、弹出后清空、全程无戳(旧 native 二进制)
|
| 25 |
+
不建 _ns 列
|
| 26 |
退出码 0 = 全部通过。
|
| 27 |
"""
|
| 28 |
|
|
|
|
| 199 |
check("features 无任何 slam 键",
|
| 200 |
not [k for k in read_features(os.path.join(tmp, "none"))
|
| 201 |
if "slam_" in k])
|
| 202 |
+
|
| 203 |
+
print("[6] slam_trajectory_ns = 逐点并行宿主时刻(list<int64>,非标量)")
|
| 204 |
+
# 分支顺序陷阱:slam_trajectory_ns 也以 _ns 结尾,若排在通用 _ns
|
| 205 |
+
# 分支之后就会被落成"每行一个标量 int64"——一行 2 个点只剩 1 个
|
| 206 |
+
# 时刻,与点列表错位且下游无从察觉。这里用类型把它钉死。
|
| 207 |
+
NS2 = [287607702508638, 287607702575000]
|
| 208 |
+
w6 = build(os.path.join(tmp, "trajns"), [
|
| 209 |
+
{"slam_trajectory": TRAJ, "slam_trajectory_ns": NS2},
|
| 210 |
+
{}, # 空窗口 → []
|
| 211 |
+
{"slam_trajectory": TRAJ[:8],
|
| 212 |
+
"slam_trajectory_ns": [287607702641000]}, # 单点窗口
|
| 213 |
+
])
|
| 214 |
+
w6._write_data_parquet()
|
| 215 |
+
w6._write_info_json()
|
| 216 |
+
colns = "observation.slam_trajectory_ns"
|
| 217 |
+
schema6 = read_schema(os.path.join(tmp, "trajns"))
|
| 218 |
+
check("列类型 = list<int64>(不是标量 int64)",
|
| 219 |
+
schema6.field(colns).type == pa.list_(pa.int64()),
|
| 220 |
+
str(schema6.field(colns).type))
|
| 221 |
+
check("确实是 list 而非标量 —— 分支顺序未被通用 _ns 抢先",
|
| 222 |
+
not schema6.field(colns).type.equals(pa.int64()))
|
| 223 |
+
gotns = read_col(os.path.join(tmp, "trajns"), colns)
|
| 224 |
+
check("多点点值逐位往返(两个点两个时刻)",
|
| 225 |
+
gotns[0] == NS2, str(gotns[0]))
|
| 226 |
+
check("无样本帧是 [] 而不是 0", gotns[1] == [], str(gotns[1]))
|
| 227 |
+
check("单点窗口 = 1 值", gotns[2] == [287607702641000], str(gotns[2]))
|
| 228 |
+
# 与 slam_trajectory 同序等长是本列的**全部意义**,逐行核对
|
| 229 |
+
traj6 = read_col(os.path.join(tmp, "trajns"),
|
| 230 |
+
"observation.slam_trajectory")
|
| 231 |
+
check("逐行与 slam_trajectory 等长(下标即配对)",
|
| 232 |
+
all(len(a) // 8 == len(b) for a, b in zip(traj6, gotns)),
|
| 233 |
+
str([(len(a) // 8, len(b)) for a, b in zip(traj6, gotns)]))
|
| 234 |
+
feats6 = read_features(os.path.join(tmp, "trajns"))[colns]
|
| 235 |
+
check("features dtype=int64 / shape=[1]",
|
| 236 |
+
feats6["dtype"] == "int64" and feats6["shape"] == [1],
|
| 237 |
+
str(feats6))
|
| 238 |
+
check("features encoding=flat_parallel_to_slam_trajectory",
|
| 239 |
+
feats6.get("encoding") == "flat_parallel_to_slam_trajectory",
|
| 240 |
+
str(feats6))
|
| 241 |
+
|
| 242 |
+
print("[7] pipeline 侧:时刻与点锁步累积、无戳补 0、全无戳不建列")
|
| 243 |
+
from core.pipeline import CameraPipeline # noqa: E402
|
| 244 |
+
p7 = CameraPipeline()
|
| 245 |
+
p7._writer = object() # 非 None 即可(write_slam_trajectory 只判非空)
|
| 246 |
+
p7._recording = True
|
| 247 |
+
p7.write_slam_trajectory(POSE7, timestamp=10.0,
|
| 248 |
+
host_ns=287607702508638)
|
| 249 |
+
p7.write_slam_trajectory(POSE7, timestamp=10.02) # 无戳
|
| 250 |
+
p7.write_slam_trajectory(POSE7, timestamp=10.04,
|
| 251 |
+
host_ns=287607702575000)
|
| 252 |
+
snap7 = p7._pop_gripper_snapshots()
|
| 253 |
+
check("三点窗口点/刻长度同步",
|
| 254 |
+
len(snap7["slam_trajectory"]) // 8 == 3
|
| 255 |
+
and len(snap7["slam_trajectory_ns"]) == 3,
|
| 256 |
+
f"{len(snap7['slam_trajectory']) // 8} vs "
|
| 257 |
+
f"{len(snap7['slam_trajectory_ns'])}")
|
| 258 |
+
check("缺戳点补 0 而不是被跳过(否则后续点全部错位)",
|
| 259 |
+
snap7["slam_trajectory_ns"] == [287607702508638, 0,
|
| 260 |
+
287607702575000],
|
| 261 |
+
str(snap7["slam_trajectory_ns"]))
|
| 262 |
+
check("弹出后累加器清空",
|
| 263 |
+
p7._pop_gripper_snapshots() == {}
|
| 264 |
+
and not p7._gripper_traj_ns)
|
| 265 |
+
p7b = CameraPipeline()
|
| 266 |
+
p7b._writer = object()
|
| 267 |
+
p7b._recording = True
|
| 268 |
+
p7b.write_slam_trajectory(POSE7, timestamp=1.0) # 全程无戳
|
| 269 |
+
snap7b = p7b._pop_gripper_snapshots()
|
| 270 |
+
check("旧二进制(全程无戳)不建 _ns 列",
|
| 271 |
+
"slam_trajectory" in snap7b
|
| 272 |
+
and "slam_trajectory_ns" not in snap7b,
|
| 273 |
+
str(sorted(snap7b)))
|
| 274 |
finally:
|
| 275 |
shutil.rmtree(tmp, ignore_errors=True)
|
| 276 |
|
collector/ui/lite_window.py
CHANGED
|
@@ -6,10 +6,17 @@
|
|
| 6 |
qt_material/mediapipe 及 ui.main_window 等),导入闭包由
|
| 7 |
tools/tests/lite_import_guard_test.py 断言守护。
|
| 8 |
|
| 9 |
-
设备范围(每类 1 台):
|
| 10 |
- D435 深度相机: RGB(1280×720) + 深度(848×480) 两路录制
|
|
|
|
| 11 |
- USB-C 手套: UsbGloveEngine(触觉 + IMU 四元数,无骨架解算)
|
| 12 |
- BLE 手套: SensorBLEEngine(触觉)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 13 |
"""
|
| 14 |
|
| 15 |
from __future__ import annotations
|
|
@@ -39,11 +46,14 @@ from core.d435_camera import D435Worker, list_d400_devices
|
|
| 39 |
from core.d435_manager import D435DeviceManager
|
| 40 |
from core.device_detector import (DeviceInfo, _list_ble_devices,
|
| 41 |
_list_d435_devices,
|
|
|
|
| 42 |
_list_usb_glove_devices,
|
| 43 |
_list_uvc_devices,
|
| 44 |
set_ble_scan_suppressed,
|
| 45 |
usb_glove_prefer_side)
|
| 46 |
from core.encoder_probe import list_working_ffmpegs
|
|
|
|
|
|
|
| 47 |
from core.helpers import delete_pooled_episode, format_duration
|
| 48 |
from core.pipeline import CameraPipeline
|
| 49 |
from core.uploader import UploadManager
|
|
@@ -63,6 +73,7 @@ LITE_PREVIEW_W, LITE_PREVIEW_H = 640, 360
|
|
| 63 |
LITE_UVC_W, LITE_UVC_H = 640, 480 # UVC 采集分辨率(低端机友好)
|
| 64 |
LITE_UVC_FPS = 30 # UVC 采集帧率(=录制帧率);预览刷新仍限 15
|
| 65 |
LITE_UVC_SLOT = "uvc_rgb" # UVC 固定槽名(与 d435_rgb 同规格)
|
|
|
|
| 66 |
|
| 67 |
# 内置暗色 QSS(替代 qt-material 主题,~20 行)
|
| 68 |
LITE_QSS = f"""
|
|
@@ -356,15 +367,19 @@ class LiteWindow(QMainWindow):
|
|
| 356 |
|
| 357 |
构造注入点(与 MainWindow 同口径,供离线测试替身):
|
| 358 |
LiteWindow(pipeline=None, upload_manager=None, d435_worker_cls=None,
|
| 359 |
-
uvc_capture=None)
|
| 360 |
"""
|
| 361 |
|
| 362 |
-
|
|
|
|
| 363 |
|
| 364 |
def __init__(self, pipeline: Optional[CameraPipeline] = None,
|
| 365 |
upload_manager: Optional[LiteUploadManager] = None,
|
| 366 |
-
d435_worker_cls=None, uvc_capture=None
|
|
|
|
| 367 |
# uvc_capture: 测试注入的 cv2.VideoCapture 替身(真实环境为 None)
|
|
|
|
|
|
|
| 368 |
super().__init__()
|
| 369 |
self.setWindowTitle(tr("DAQ 极简采集 (lite v{})", settings.APP_VERSION))
|
| 370 |
|
|
@@ -376,6 +391,7 @@ class LiteWindow(QMainWindow):
|
|
| 376 |
self._d435_worker_cls = d435_worker_cls or D435Worker
|
| 377 |
self._d435_manager = D435DeviceManager(self._pipeline)
|
| 378 |
self._uvc_capture = uvc_capture
|
|
|
|
| 379 |
|
| 380 |
# 设备注册表: dev.key → d435 entry / {"kind": glove, "pump": …,
|
| 381 |
# "engine": …, "role": …}
|
|
@@ -445,7 +461,8 @@ class LiteWindow(QMainWindow):
|
|
| 445 |
("d435", tr("D435 深度相机"), ["_d435_key"]),
|
| 446 |
("uvc", tr("UVC 摄像头"), ["_uvc_key"]),
|
| 447 |
("usb", tr("USB-C 手套"), ["_usb_key"]),
|
| 448 |
-
("ble", tr("BLE 手套"), ["_ble_key", "_ble_key2"])
|
|
|
|
| 449 |
grp = QGroupBox(title)
|
| 450 |
gv = QVBoxLayout(grp)
|
| 451 |
rows = []
|
|
@@ -577,16 +594,27 @@ class LiteWindow(QMainWindow):
|
|
| 577 |
usb = _list_usb_glove_devices()
|
| 578 |
ble = _list_ble_devices()
|
| 579 |
uvc = _list_uvc_devices()
|
| 580 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 581 |
finally:
|
| 582 |
self._scan_busy = False
|
| 583 |
|
| 584 |
def _on_scan_result(self, d435: List[DeviceInfo],
|
| 585 |
usb: List[DeviceInfo], ble: List[DeviceInfo],
|
| 586 |
-
uvc: List[DeviceInfo]
|
|
|
|
| 587 |
"""扫描结果回填设备下拉框(保留当前选中/已开启项)。"""
|
| 588 |
for kind, devs in (("d435", d435), ("usb", usb), ("ble", ble),
|
| 589 |
-
("uvc", uvc)):
|
| 590 |
for row in self._dev_groups[kind]:
|
| 591 |
combo = row["combo"]
|
| 592 |
open_key = getattr(self, row["attr"]) or ""
|
|
@@ -628,7 +656,8 @@ class LiteWindow(QMainWindow):
|
|
| 628 |
openers = {"d435": self._open_d435,
|
| 629 |
"uvc": self._open_uvc,
|
| 630 |
"usb": self._open_usb_glove,
|
| 631 |
-
"ble": lambda d: self._open_ble_glove(d, attr)
|
|
|
|
| 632 |
if openers[kind](dev):
|
| 633 |
row["btn"].setText(tr("关闭"))
|
| 634 |
self._log(tr("[设备] 已开启 {}", dev.label))
|
|
@@ -660,6 +689,9 @@ class LiteWindow(QMainWindow):
|
|
| 660 |
self._pipeline.unregister_external_source(entry["slots"][0])
|
| 661 |
self._ring.pop(entry["slots"][0], None)
|
| 662 |
self._recover_preview_source(entry["slots"][0])
|
|
|
|
|
|
|
|
|
|
| 663 |
else:
|
| 664 |
pump = entry["pump"]
|
| 665 |
try:
|
|
@@ -838,6 +870,378 @@ class LiteWindow(QMainWindow):
|
|
| 838 |
dev.serial or dev.label, role))
|
| 839 |
return True
|
| 840 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 841 |
# ── 录制控制 ──────────────────────────────────────
|
| 842 |
|
| 843 |
def _device_meta(self) -> List[dict]:
|
|
@@ -851,25 +1255,64 @@ class LiteWindow(QMainWindow):
|
|
| 851 |
meta.append({"key": key, "kind": "uvc", "name": e["label"],
|
| 852 |
"serial": e.get("serial", ""),
|
| 853 |
"slots": e["slots"]})
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 854 |
else:
|
| 855 |
meta.append({"key": key, "kind": e["kind"],
|
| 856 |
"name": e["label"], "serial": "",
|
| 857 |
"slots": [e["role"]]})
|
| 858 |
return meta
|
| 859 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 860 |
def _start_recording(self):
|
| 861 |
if self._pipeline.is_recording:
|
| 862 |
return
|
| 863 |
-
# pipeline.py:
|
| 864 |
-
#
|
| 865 |
-
|
| 866 |
-
|
| 867 |
-
|
| 868 |
-
|
| 869 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 870 |
return
|
| 871 |
-
|
| 872 |
-
else LITE_UVC_SLOT)
|
| 873 |
self._pipeline.start_recording(
|
| 874 |
master,
|
| 875 |
task_name=self._task_edit.text().strip(),
|
|
@@ -885,6 +1328,9 @@ class LiteWindow(QMainWindow):
|
|
| 885 |
self._pipeline.abort_recording("")
|
| 886 |
|
| 887 |
def _on_recording_started(self, slot_id: str):
|
|
|
|
|
|
|
|
|
|
| 888 |
self._rec_status.setText(tr("⏺ 录制中"))
|
| 889 |
self._btn_start.setEnabled(False)
|
| 890 |
self._btn_stop.setEnabled(True)
|
|
@@ -892,6 +1338,11 @@ class LiteWindow(QMainWindow):
|
|
| 892 |
self._set_device_buttons_enabled(False)
|
| 893 |
|
| 894 |
def _on_recording_finished(self, slot_id: str, session_path: str):
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 895 |
self._rec_status.setText(tr("待机"))
|
| 896 |
self._btn_start.setEnabled(True)
|
| 897 |
self._btn_stop.setEnabled(False)
|
|
@@ -924,6 +1375,10 @@ class LiteWindow(QMainWindow):
|
|
| 924 |
os.path.basename(session_path)))
|
| 925 |
|
| 926 |
def _on_recording_aborted(self, slot_id: str):
|
|
|
|
|
|
|
|
|
|
|
|
|
| 927 |
self._rec_status.setText(tr("待机"))
|
| 928 |
self._btn_start.setEnabled(True)
|
| 929 |
self._btn_stop.setEnabled(False)
|
|
@@ -1053,7 +1508,8 @@ class LiteWindow(QMainWindow):
|
|
| 1053 |
|
| 1054 |
def closeEvent(self, event):
|
| 1055 |
self._scan_timer.stop()
|
| 1056 |
-
|
|
|
|
| 1057 |
for i, row in enumerate(self._dev_groups[kind]):
|
| 1058 |
if getattr(self, row["attr"]):
|
| 1059 |
try:
|
|
|
|
| 6 |
qt_material/mediapipe 及 ui.main_window 等),导入闭包由
|
| 7 |
tools/tests/lite_import_guard_test.py 断言守护。
|
| 8 |
|
| 9 |
+
设备范围(每类 1 台;夹爪最多 2 台):
|
| 10 |
- D435 深度相机: RGB(1280×720) + 深度(848×480) 两路录制
|
| 11 |
+
- UVC 摄像头: 单路 RGB 录制
|
| 12 |
- USB-C 手套: UsbGloveEngine(触觉 + IMU 四元数,无骨架解算)
|
| 13 |
- BLE 手套: SensorBLEEngine(触觉)
|
| 14 |
+
- UMI 夹爪: GripperBridge 全链(RGB 录制 + 触觉力/力矩阵 + SLAM
|
| 15 |
+
轨迹),**只录不显** —— 无热力图/轨迹/开合板控件,
|
| 16 |
+
预览区与主程序无夹爪时完全一致。原生资源
|
| 17 |
+
(core/gripper/native,~470MB)缺失时扫描不到设备:
|
| 18 |
+
Windows 包不带该资源,于是五组设备框照常都在(和没插
|
| 19 |
+
D435 时一样),「UMI 夹爪」那一组里面是空的。
|
| 20 |
"""
|
| 21 |
|
| 22 |
from __future__ import annotations
|
|
|
|
| 46 |
from core.d435_manager import D435DeviceManager
|
| 47 |
from core.device_detector import (DeviceInfo, _list_ble_devices,
|
| 48 |
_list_d435_devices,
|
| 49 |
+
_list_gripper_devices,
|
| 50 |
_list_usb_glove_devices,
|
| 51 |
_list_uvc_devices,
|
| 52 |
set_ble_scan_suppressed,
|
| 53 |
usb_glove_prefer_side)
|
| 54 |
from core.encoder_probe import list_working_ffmpegs
|
| 55 |
+
from core.gripper_codec import (describe_gripper_matrix_spec,
|
| 56 |
+
encode_gripper_force_matrix)
|
| 57 |
from core.helpers import delete_pooled_episode, format_duration
|
| 58 |
from core.pipeline import CameraPipeline
|
| 59 |
from core.uploader import UploadManager
|
|
|
|
| 73 |
LITE_UVC_W, LITE_UVC_H = 640, 480 # UVC 采集分辨率(低端机友好)
|
| 74 |
LITE_UVC_FPS = 30 # UVC 采集帧率(=录制帧率);预览刷新仍限 15
|
| 75 |
LITE_UVC_SLOT = "uvc_rgb" # UVC 固定槽名(与 d435_rgb 同规格)
|
| 76 |
+
LITE_GRIPPER_MAX = 2 # 同时开启的夹爪上限(rig1/rig2)
|
| 77 |
|
| 78 |
# 内置暗色 QSS(替代 qt-material 主题,~20 行)
|
| 79 |
LITE_QSS = f"""
|
|
|
|
| 367 |
|
| 368 |
构造注入点(与 MainWindow 同口径,供离线测试替身):
|
| 369 |
LiteWindow(pipeline=None, upload_manager=None, d435_worker_cls=None,
|
| 370 |
+
uvc_capture=None, gripper_bridge_cls=None)
|
| 371 |
"""
|
| 372 |
|
| 373 |
+
# (d435, usb, ble, uvc, gripper)
|
| 374 |
+
_scan_result = pyqtSignal(list, list, list, list, list)
|
| 375 |
|
| 376 |
def __init__(self, pipeline: Optional[CameraPipeline] = None,
|
| 377 |
upload_manager: Optional[LiteUploadManager] = None,
|
| 378 |
+
d435_worker_cls=None, uvc_capture=None,
|
| 379 |
+
gripper_bridge_cls=None):
|
| 380 |
# uvc_capture: 测试注入的 cv2.VideoCapture 替身(真实环境为 None)
|
| 381 |
+
# gripper_bridge_cls: 测试注入的假桥(非 None 时跳过资源闸门,
|
| 382 |
+
# 与 d435_worker_cls 同口径)
|
| 383 |
super().__init__()
|
| 384 |
self.setWindowTitle(tr("DAQ 极简采集 (lite v{})", settings.APP_VERSION))
|
| 385 |
|
|
|
|
| 391 |
self._d435_worker_cls = d435_worker_cls or D435Worker
|
| 392 |
self._d435_manager = D435DeviceManager(self._pipeline)
|
| 393 |
self._uvc_capture = uvc_capture
|
| 394 |
+
self._gripper_bridge_cls = gripper_bridge_cls
|
| 395 |
|
| 396 |
# 设备注册表: dev.key → d435 entry / {"kind": glove, "pump": …,
|
| 397 |
# "engine": …, "role": …}
|
|
|
|
| 461 |
("d435", tr("D435 深度相机"), ["_d435_key"]),
|
| 462 |
("uvc", tr("UVC 摄像头"), ["_uvc_key"]),
|
| 463 |
("usb", tr("USB-C 手套"), ["_usb_key"]),
|
| 464 |
+
("ble", tr("BLE 手套"), ["_ble_key", "_ble_key2"]),
|
| 465 |
+
("gripper", tr("UMI 夹爪"), ["_gripper_key"])):
|
| 466 |
grp = QGroupBox(title)
|
| 467 |
gv = QVBoxLayout(grp)
|
| 468 |
rows = []
|
|
|
|
| 594 |
usb = _list_usb_glove_devices()
|
| 595 |
ble = _list_ble_devices()
|
| 596 |
uvc = _list_uvc_devices()
|
| 597 |
+
# 夹爪:_list_gripper_devices 内部先查 paths.gripper_resources_available()
|
| 598 |
+
# (native/ 未随包交付 → 空列表 → 组框照建、里面空着;Windows 上
|
| 599 |
+
# core.gripper 因顶层 import fcntl 直接导入失败,同样被它内部
|
| 600 |
+
# 的 except 兜成空列表)
|
| 601 |
+
gripper = _list_gripper_devices()
|
| 602 |
+
try:
|
| 603 |
+
self._scan_result.emit(d435, usb, ble, uvc, gripper)
|
| 604 |
+
except RuntimeError:
|
| 605 |
+
# 窗口已销毁(关窗时扫描线程仍在跑)—— 投递不出去是预期,
|
| 606 |
+
# 不必在测试/生产日志里留下吓人的线程回溯
|
| 607 |
+
pass
|
| 608 |
finally:
|
| 609 |
self._scan_busy = False
|
| 610 |
|
| 611 |
def _on_scan_result(self, d435: List[DeviceInfo],
|
| 612 |
usb: List[DeviceInfo], ble: List[DeviceInfo],
|
| 613 |
+
uvc: List[DeviceInfo],
|
| 614 |
+
gripper: List[DeviceInfo] = None):
|
| 615 |
"""扫描结果回填设备下拉框(保留当前选中/已开启项)。"""
|
| 616 |
for kind, devs in (("d435", d435), ("usb", usb), ("ble", ble),
|
| 617 |
+
("uvc", uvc), ("gripper", gripper or [])):
|
| 618 |
for row in self._dev_groups[kind]:
|
| 619 |
combo = row["combo"]
|
| 620 |
open_key = getattr(self, row["attr"]) or ""
|
|
|
|
| 656 |
openers = {"d435": self._open_d435,
|
| 657 |
"uvc": self._open_uvc,
|
| 658 |
"usb": self._open_usb_glove,
|
| 659 |
+
"ble": lambda d: self._open_ble_glove(d, attr),
|
| 660 |
+
"gripper": self._open_gripper}
|
| 661 |
if openers[kind](dev):
|
| 662 |
row["btn"].setText(tr("关闭"))
|
| 663 |
self._log(tr("[设备] 已开启 {}", dev.label))
|
|
|
|
| 689 |
self._pipeline.unregister_external_source(entry["slots"][0])
|
| 690 |
self._ring.pop(entry["slots"][0], None)
|
| 691 |
self._recover_preview_source(entry["slots"][0])
|
| 692 |
+
elif kind == "gripper":
|
| 693 |
+
# 夹爪条目没有 "pump" 键,必须排在下面 else(手套)分支之前
|
| 694 |
+
self._close_gripper(key)
|
| 695 |
else:
|
| 696 |
pump = entry["pump"]
|
| 697 |
try:
|
|
|
|
| 870 |
dev.serial or dev.label, role))
|
| 871 |
return True
|
| 872 |
|
| 873 |
+
# ── UMI 夹爪(只录不显:无网格槽/热力图/轨迹/开合板控件)──
|
| 874 |
+
#
|
| 875 |
+
# lite 版 = main_window 夹爪链的裁剪:**数据链一字不改**(槽名、数据列、
|
| 876 |
+
# 桥接信号、力矩阵泵、CPU 预留),只删控件相关(网格槽、PoseViewQt、
|
| 877 |
+
# GripperSideWidget、双目显示节流)与 S80M 冲突检查(lite 无 S80M)。
|
| 878 |
+
# 落盘能力全在 core.pipeline / core.egodata_writer,已数据驱动。
|
| 879 |
+
|
| 880 |
+
@staticmethod
|
| 881 |
+
def _gripper_slot_map(index: int) -> dict:
|
| 882 |
+
"""夹爪槽位表:rig1 保持旧槽名,rig2 加序号。
|
| 883 |
+
|
| 884 |
+
与 main_window._gripper_slot_map 逐字相同 —— 槽名是下游数据契约
|
| 885 |
+
(视频目录名 / parquet 列前缀),lite 与主程序录出的 episode 必须
|
| 886 |
+
同名,改这里等于改数据格式。
|
| 887 |
+
"""
|
| 888 |
+
mid = "" if index == 1 else f"_{index}"
|
| 889 |
+
return {
|
| 890 |
+
"stereo_l": f"gripper{mid}_stereo_left",
|
| 891 |
+
"rgb": f"gripper{mid}_rgb",
|
| 892 |
+
"pose": f"gripper{mid}_pose",
|
| 893 |
+
"force_l": f"gripper{mid}_force_left",
|
| 894 |
+
"force_r": f"gripper{mid}_force_right",
|
| 895 |
+
}
|
| 896 |
+
|
| 897 |
+
def _open_gripper(self, dev: DeviceInfo) -> bool:
|
| 898 |
+
"""开启 UMI 夹爪:建数据链(bridge + 槽注册 + CPU 预留),无画面控件。
|
| 899 |
+
|
| 900 |
+
双目槽只交给桥接做 SLAM 解算,不注册外部帧源 → 不产生视频、不落盘;
|
| 901 |
+
RGB 槽由首帧惰性注册(分辨率以实际首帧为准)。
|
| 902 |
+
双夹爪:打开顺序分配 rig 序号(1..2,关闭即释放),槽名/数据列
|
| 903 |
+
rig1 保持旧契约、rig2 带 "gripper_2_" 前缀。
|
| 904 |
+
"""
|
| 905 |
+
if dev.key in self._workers:
|
| 906 |
+
return True # 已开启(幂等)
|
| 907 |
+
gripper_count = sum(
|
| 908 |
+
1 for e in self._workers.values() if e.get("kind") == "gripper")
|
| 909 |
+
if gripper_count >= LITE_GRIPPER_MAX:
|
| 910 |
+
QMessageBox.warning(
|
| 911 |
+
self, tr("夹爪数量上限"),
|
| 912 |
+
tr("最多支持同时开启 {} 台 UMI 夹爪。\n"
|
| 913 |
+
"请先关闭其中一台再开启。", LITE_GRIPPER_MAX))
|
| 914 |
+
self._log(tr("[设备] 夹爪数量已达上限,已拒绝开启: {}", dev.label))
|
| 915 |
+
return False
|
| 916 |
+
index = 1 if gripper_count == 0 else 2
|
| 917 |
+
# 资源闸门:native/ 未随包交付时拒绝开启(Windows 包即此路径)。
|
| 918 |
+
# 注入假桥时跳过 —— 与 _open_d435 的 worker_cls 口径一致
|
| 919 |
+
if self._gripper_bridge_cls is None:
|
| 920 |
+
try:
|
| 921 |
+
from core.gripper import paths
|
| 922 |
+
if not paths.gripper_resources_available():
|
| 923 |
+
QMessageBox.warning(
|
| 924 |
+
self, tr("夹爪资源缺失"),
|
| 925 |
+
tr("未找到夹爪原生资源(core/gripper/native 未随包"
|
| 926 |
+
"交付),无法开启。"))
|
| 927 |
+
return False
|
| 928 |
+
except Exception:
|
| 929 |
+
return False
|
| 930 |
+
if not dev.serial:
|
| 931 |
+
QMessageBox.warning(
|
| 932 |
+
self, tr("夹爪无序列号"),
|
| 933 |
+
tr("未读取到夹爪 ESP32 序列号,无法唯一定位控制板。"))
|
| 934 |
+
return False
|
| 935 |
+
|
| 936 |
+
slot_map = self._gripper_slot_map(index)
|
| 937 |
+
bridge_cls = self._gripper_bridge_cls
|
| 938 |
+
if bridge_cls is None:
|
| 939 |
+
from core.gripper.bridge import GripperBridge
|
| 940 |
+
bridge_cls = GripperBridge
|
| 941 |
+
bridge = bridge_cls(self, stereo_slot=slot_map["stereo_l"],
|
| 942 |
+
rig_index=index)
|
| 943 |
+
key = dev.key
|
| 944 |
+
# 帧信号按 dev.key lambda 捕获路由(双夹爪各自归位)。
|
| 945 |
+
# 桥接还有 stereo_frame_ready —— lite 不显示双目,故不接线(丢帧
|
| 946 |
+
# 统计在桥接内部 _stereo_drop_tick,录制结束由 _gripper_drop_summary 取)。
|
| 947 |
+
bridge.rgb_frame_ready.connect(
|
| 948 |
+
lambda frame, hw_ns, key=key:
|
| 949 |
+
self._on_gripper_rgb(key, frame, hw_ns))
|
| 950 |
+
bridge.tactile_ready.connect(
|
| 951 |
+
lambda side, heatmap, force, matrix, capture_ns, key=key:
|
| 952 |
+
self._on_gripper_tactile(key, side, heatmap, force, matrix,
|
| 953 |
+
capture_ns))
|
| 954 |
+
bridge.pose_ready.connect(
|
| 955 |
+
lambda position, quat, trajectory=(), timestamp=None,
|
| 956 |
+
host_ns=None, key=key:
|
| 957 |
+
self._on_gripper_pose(key, position, quat, trajectory,
|
| 958 |
+
timestamp, host_ns))
|
| 959 |
+
bridge.gripper_state_ready.connect(
|
| 960 |
+
lambda payload, key=key: self._on_gripper_state(key, payload))
|
| 961 |
+
bridge.log.connect(self._log)
|
| 962 |
+
bridge.error.connect(lambda msg, key=key: self._on_gripper_error(key, msg))
|
| 963 |
+
bridge.opened.connect(
|
| 964 |
+
lambda key=key: self._log(
|
| 965 |
+
tr("[夹爪] {} 相机 + 触觉 + SLAM 链路已就绪",
|
| 966 |
+
self._workers.get(key, {}).get("label", key))))
|
| 967 |
+
entry = {"kind": "gripper", "label": dev.label,
|
| 968 |
+
"serial": dev.serial or "",
|
| 969 |
+
"slots": [slot_map["stereo_l"], slot_map["rgb"],
|
| 970 |
+
slot_map["pose"], slot_map["force_l"],
|
| 971 |
+
slot_map["force_r"]],
|
| 972 |
+
"bridge": bridge, "fays_serial": "", "sensor_columns": [],
|
| 973 |
+
"_video_registered": set(), # RGB 槽惰性注册标记
|
| 974 |
+
"matrix_pump_stop": None, # 录制期力矩阵泵停止事件
|
| 975 |
+
"gripper_index": index, "slot_map": slot_map,
|
| 976 |
+
# 双夹爪数据列命名空间:rig1 空串(旧键 slam_trajectory/
|
| 977 |
+
# gripper_state/gripper_{side}_force…不变),rig2 加前缀
|
| 978 |
+
"snapshot_prefix": ("" if index == 1
|
| 979 |
+
else f"gripper_{index}_")}
|
| 980 |
+
self._workers[dev.key] = entry
|
| 981 |
+
# 非视频槽(pose/左右触觉)单独进 status 列
|
| 982 |
+
for sid in (slot_map["pose"], slot_map["force_l"], slot_map["force_r"]):
|
| 983 |
+
self._pipeline.register_gripper_device_id(sid)
|
| 984 |
+
self._pipeline.record_event(dev.key, "connected")
|
| 985 |
+
# CPU 预留(主线程掩码收窄,让本 rig 的 SLAM 分区与 raw 专用核不再被
|
| 986 |
+
# 主进程线程/ffmpeg 子进程抢占)——必须在 open() 派生各工作线程之前
|
| 987 |
+
# 生效(子线程继承受限掩码;native SLAM 走 taskset 显式设掩码不受影响)
|
| 988 |
+
try:
|
| 989 |
+
from core.gripper import affinity
|
| 990 |
+
affinity.reserve(index, self._log)
|
| 991 |
+
except Exception as exc: # 预留失败不该拦下设备(lite 兜底)
|
| 992 |
+
self._log(tr("[夹爪] CPU 预留失败(忽略): {}", exc))
|
| 993 |
+
bridge.open(dev.serial)
|
| 994 |
+
self._gripper_key = dev.key
|
| 995 |
+
self._log(tr("[夹爪] {} 正在启动相机/触觉/SLAM 链路…(只录不显)",
|
| 996 |
+
dev.label))
|
| 997 |
+
return True
|
| 998 |
+
|
| 999 |
+
def _close_gripper(self, dev_key: str):
|
| 1000 |
+
"""关闭 UMI 夹爪:停泵 → 桥接逆序回收 → 注销数据列 → 退 CPU 预留。
|
| 1001 |
+
|
| 1002 |
+
槽位全部取自本条目 slot_map(rig2 前缀不同),关闭一台不碰另一台
|
| 1003 |
+
的共享命名空间。lite 无网格槽/轨迹控件,故不删控件。
|
| 1004 |
+
"""
|
| 1005 |
+
entry = self._workers.pop(dev_key, None)
|
| 1006 |
+
if not entry or entry["kind"] != "gripper":
|
| 1007 |
+
return
|
| 1008 |
+
rig_index = entry.get("gripper_index") or 1
|
| 1009 |
+
self._stop_matrix_pump(entry)
|
| 1010 |
+
bridge = entry.get("bridge")
|
| 1011 |
+
if bridge is not None:
|
| 1012 |
+
try:
|
| 1013 |
+
bridge.close()
|
| 1014 |
+
except Exception as exc:
|
| 1015 |
+
self._log(tr("[夹爪] 关闭桥接异常(忽略): {}", exc))
|
| 1016 |
+
# 退还本 rig 的 CPU 预留(掩码扩回;全部 rig 关闭后恢复完整掩码)
|
| 1017 |
+
try:
|
| 1018 |
+
from core.gripper import affinity
|
| 1019 |
+
affinity.release(rig_index, self._log)
|
| 1020 |
+
except Exception:
|
| 1021 |
+
pass
|
| 1022 |
+
slot_map = entry["slot_map"]
|
| 1023 |
+
# 双目槽只供 SLAM 解算(从未注册外部帧源),只有 RGB 需要注销
|
| 1024 |
+
self._pipeline.unregister_external_source(slot_map["rgb"])
|
| 1025 |
+
for sid in (slot_map["pose"], slot_map["force_l"],
|
| 1026 |
+
slot_map["force_r"]):
|
| 1027 |
+
self._pipeline.unregister_gripper_device_id(sid)
|
| 1028 |
+
self._pipeline.record_event(dev_key, "disconnected")
|
| 1029 |
+
# 直接调用本函数(错误路径/测试)时也要摘掉开着的标记,
|
| 1030 |
+
# 否则面板按钮停在「关闭」、closeEvent 再也收不到这台
|
| 1031 |
+
if self._gripper_key == dev_key:
|
| 1032 |
+
self._gripper_key = ""
|
| 1033 |
+
|
| 1034 |
+
# ── 夹爪桥接信号(dev.key 路由,双夹爪各自归位)──
|
| 1035 |
+
|
| 1036 |
+
def _gripper_entry(self, dev_key: str):
|
| 1037 |
+
entry = self._workers.get(dev_key)
|
| 1038 |
+
if entry is None or entry.get("kind") != "gripper":
|
| 1039 |
+
return None
|
| 1040 |
+
return entry
|
| 1041 |
+
|
| 1042 |
+
def _gripper_entries(self) -> List[dict]:
|
| 1043 |
+
"""全部已开启夹爪条目(双夹爪录制时逐台处理)。"""
|
| 1044 |
+
return [e for e in self._workers.values()
|
| 1045 |
+
if e.get("kind") == "gripper"]
|
| 1046 |
+
|
| 1047 |
+
def _on_gripper_rgb(self, dev_key: str, frame, hw_ns: int):
|
| 1048 |
+
"""DECXIN RGB 帧(bridge 线程 → 队列回主线程):首帧惰性注册外部
|
| 1049 |
+
帧源(分辨率以实际首帧为准,帧率与主程序同口径 30),录制时写入
|
| 1050 |
+
本 rig 的 RGB 视频槽。lite 无画面控件,帧不进预览也不显示。"""
|
| 1051 |
+
entry = self._gripper_entry(dev_key)
|
| 1052 |
+
if entry is None:
|
| 1053 |
+
return
|
| 1054 |
+
sid = entry["slot_map"]["rgb"]
|
| 1055 |
+
if sid not in entry["_video_registered"]:
|
| 1056 |
+
h, w = frame.shape[:2]
|
| 1057 |
+
self._pipeline.register_external_source(sid, (h, w), fps=30.0)
|
| 1058 |
+
entry["_video_registered"].add(sid)
|
| 1059 |
+
self._log(tr("[夹爪] RGB 外部帧源已注册 {}x{}@30", w, h))
|
| 1060 |
+
if self._pipeline.is_recording:
|
| 1061 |
+
self._pipeline.write_external_frame(
|
| 1062 |
+
sid, frame.copy(), hardware_ns=hw_ns)
|
| 1063 |
+
|
| 1064 |
+
def _on_gripper_pose(self, dev_key: str, position, quat, trajectory=(),
|
| 1065 |
+
timestamp=None, host_ns=None):
|
| 1066 |
+
"""SLAM 位姿(pos3, quat4, traj, t, host_ns)→ 轨迹点落盘(rig2 加前缀)。
|
| 1067 |
+
|
| 1068 |
+
轨迹并入 episode parquet,无 txt 侧车;t 与 host_ns 进
|
| 1069 |
+
slam_trajectory / slam_trajectory_ns 列。位姿本身不落 slam_pose
|
| 1070 |
+
列(2026-09-10 定案)。lite 无轨迹显示控件,trajectory/position/
|
| 1071 |
+
quat 只用于拼落盘行。host_ns = 该取样帧的宿主单调钟纳秒(与
|
| 1072 |
+
hardware_ns 同时基),是 slam 点与视频帧按时间对齐的依据。
|
| 1073 |
+
"""
|
| 1074 |
+
entry = self._gripper_entry(dev_key)
|
| 1075 |
+
if entry is None:
|
| 1076 |
+
return
|
| 1077 |
+
qx, qy, qz, qw = (float(value) for value in quat)
|
| 1078 |
+
self._pipeline.write_slam_trajectory(
|
| 1079 |
+
[float(position[0]), float(position[1]), float(position[2]),
|
| 1080 |
+
qx, qy, qz, qw],
|
| 1081 |
+
prefix=entry["snapshot_prefix"], timestamp=timestamp,
|
| 1082 |
+
host_ns=host_ns)
|
| 1083 |
+
|
| 1084 |
+
def _on_gripper_tactile(self, dev_key: str, side, heatmap, force,
|
| 1085 |
+
force_matrix, capture_ns=0):
|
| 1086 |
+
"""触觉结果(左/右):力值落盘;力矩阵由 P4 泵线程各自落盘。
|
| 1087 |
+
|
| 1088 |
+
capture_ns 是该样本的采集时刻,**必须随力值一起落盘** —— 力矩阵走
|
| 1089 |
+
独立泵线程读最新单槽(恒最新),RGB 走主线程 FIFO(较旧),两者在
|
| 1090 |
+
处理端靠这个时刻做最近邻对齐,丢了就再也对不上(见 v1.3.3 修复)。
|
| 1091 |
+
lite 无热力图控件,heatmap/force_matrix 只用于保持信号签名一致。
|
| 1092 |
+
"""
|
| 1093 |
+
entry = self._gripper_entry(dev_key)
|
| 1094 |
+
if entry is None:
|
| 1095 |
+
return
|
| 1096 |
+
if force is not None and len(force) == 3:
|
| 1097 |
+
self._pipeline.write_tactile_force(
|
| 1098 |
+
side, force, prefix=entry["snapshot_prefix"],
|
| 1099 |
+
capture_ns=capture_ns)
|
| 1100 |
+
|
| 1101 |
+
def _on_gripper_state(self, dev_key: str, payload):
|
| 1102 |
+
"""串口夹爪状态 {pct,gripped,fz,event,board} → gripper_state 稀疏列
|
| 1103 |
+
快照(录制期随帧行落盘)。lite 无开合板控件。"""
|
| 1104 |
+
entry = self._gripper_entry(dev_key)
|
| 1105 |
+
if entry is None:
|
| 1106 |
+
return
|
| 1107 |
+
pct = payload.get("pct")
|
| 1108 |
+
if pct is None:
|
| 1109 |
+
return # 板状态尚未到来:不写列(write 侧要求有限数值)
|
| 1110 |
+
gripped = bool(payload.get("gripped", False))
|
| 1111 |
+
fz = payload.get("fz", 0.0)
|
| 1112 |
+
self._pipeline.write_gripper_state(
|
| 1113 |
+
[float(pct), float(gripped), float(fz)],
|
| 1114 |
+
prefix=entry["snapshot_prefix"])
|
| 1115 |
+
|
| 1116 |
+
def _on_gripper_error(self, dev_key: str, msg: str):
|
| 1117 |
+
"""桥接启动失败:弹窗 + 回收条目 + 面板按钮回退「开启」。"""
|
| 1118 |
+
self._log(tr("[夹爪] 启动失败: {}", msg))
|
| 1119 |
+
QMessageBox.critical(self, tr("夹爪启动失败"), msg)
|
| 1120 |
+
# 先记下它是不是面板上开着的那台:_close_gripper 会把标记摘掉
|
| 1121 |
+
was_open = self._gripper_key == dev_key
|
| 1122 |
+
self._close_gripper(dev_key)
|
| 1123 |
+
if was_open:
|
| 1124 |
+
for row in self._dev_groups["gripper"]:
|
| 1125 |
+
if row["attr"] == "_gripper_key":
|
| 1126 |
+
row["btn"].setText(tr("开启"))
|
| 1127 |
+
|
| 1128 |
+
def _gripper_drop_summary(self):
|
| 1129 |
+
"""录制结束汇总夹爪双目空桶统计(30fps 口径,有丢才打日志)。
|
| 1130 |
+
|
| 1131 |
+
只录不显时这是链路健康**唯一**的可见证据:双目取帧未达 30fps 就
|
| 1132 |
+
意味着 SLAM 轨迹有缺帧,而画面上看不出来。
|
| 1133 |
+
"""
|
| 1134 |
+
for entry in self._gripper_entries():
|
| 1135 |
+
bridge = entry.get("bridge")
|
| 1136 |
+
if bridge is None:
|
| 1137 |
+
continue
|
| 1138 |
+
dropped, elapsed = bridge.stereo_drop_snapshot()
|
| 1139 |
+
if dropped > 0 and elapsed > 0:
|
| 1140 |
+
self._log(tr("[夹爪] {} 本次录制双目空桶 {} 个({:.0f}%)"
|
| 1141 |
+
"——SLAM 取帧未达 30fps,轨迹存在缺帧",
|
| 1142 |
+
entry.get("label", "UMI"),
|
| 1143 |
+
dropped, dropped / elapsed * 100))
|
| 1144 |
+
|
| 1145 |
+
# ── 力矩阵泵线程(录制期)──────────────────────────
|
| 1146 |
+
|
| 1147 |
+
def _start_matrix_pump(self):
|
| 1148 |
+
"""录制开始:为每台已开启的夹爪各起一个 30ms 力矩阵泵线程。
|
| 1149 |
+
|
| 1150 |
+
必须在 pipeline 的 start_episode 之后(recording_started 信号即
|
| 1151 |
+
在它之后 emit),否则刚登记的倍率会被 start_episode 清掉。
|
| 1152 |
+
"""
|
| 1153 |
+
started = 0
|
| 1154 |
+
for entry in self._gripper_entries():
|
| 1155 |
+
bridge = entry.get("bridge")
|
| 1156 |
+
if bridge is None or entry.get("matrix_pump_stop") is not None:
|
| 1157 |
+
continue
|
| 1158 |
+
stop = threading.Event()
|
| 1159 |
+
entry["matrix_pump_stop"] = stop
|
| 1160 |
+
entry["matrix_wrote"] = {"left": 0, "right": 0}
|
| 1161 |
+
# 规格在录制开始锁定:parquet 一列只有一种类型,录制中切换会让
|
| 1162 |
+
# 同一列出现 int/float 混型(write 侧直接报错,整段录制废掉)。
|
| 1163 |
+
spec = settings.GRIPPER_FORCE_MATRIX_SPEC
|
| 1164 |
+
entry["matrix_spec"] = spec
|
| 1165 |
+
# 倍率登记到 writer:×10/×100/×1000 落盘类型都是 int16,元素类型
|
| 1166 |
+
# 分辨不出倍率,info.json features.scale 是唯一的对外凭据。
|
| 1167 |
+
self._pipeline.set_force_matrix_spec(
|
| 1168 |
+
entry["snapshot_prefix"], spec)
|
| 1169 |
+
self._log(tr("[夹爪] 力矩阵保存规格:{}",
|
| 1170 |
+
describe_gripper_matrix_spec(spec)))
|
| 1171 |
+
thread = threading.Thread(
|
| 1172 |
+
target=self._matrix_pump_loop,
|
| 1173 |
+
args=(bridge, stop, entry),
|
| 1174 |
+
name="gripper-matrix-pump-{}".format(
|
| 1175 |
+
entry.get("gripper_index", "")),
|
| 1176 |
+
daemon=True,
|
| 1177 |
+
)
|
| 1178 |
+
thread.start()
|
| 1179 |
+
entry["matrix_pump_thread"] = thread
|
| 1180 |
+
started += 1
|
| 1181 |
+
if not started:
|
| 1182 |
+
self._log(tr("[夹爪] 力矩阵泵未启动:无夹爪设备条目"))
|
| 1183 |
+
|
| 1184 |
+
def _matrix_pump_loop(self, bridge, stop, entry):
|
| 1185 |
+
"""泵线程体:仅在有新矩阵时编码(按本次录制锁定的规格)+ 快照写入。"""
|
| 1186 |
+
last_seq = {"left": 0, "right": 0}
|
| 1187 |
+
first_logged = {"left": False, "right": False}
|
| 1188 |
+
wrote = entry["matrix_wrote"]
|
| 1189 |
+
spec = entry.get("matrix_spec",
|
| 1190 |
+
settings.GRIPPER_FORCE_MATRIX_SPEC_DEFAULT)
|
| 1191 |
+
started = time.monotonic()
|
| 1192 |
+
self._log(tr("[夹爪] 力矩阵泵已启动 (L seq={} R seq={})",
|
| 1193 |
+
bridge.matrix_seq("left"),
|
| 1194 |
+
bridge.matrix_seq("right")))
|
| 1195 |
+
while not stop.is_set():
|
| 1196 |
+
for side in ("left", "right"):
|
| 1197 |
+
seq = bridge.matrix_seq(side)
|
| 1198 |
+
if seq == last_seq[side]:
|
| 1199 |
+
continue
|
| 1200 |
+
last_seq[side] = seq
|
| 1201 |
+
# 时刻与矩阵同锁取出(latest_matrix_stamped),保证配对;
|
| 1202 |
+
# 泵线程与主线程各自落各自模态,所以矩阵的时刻单独记
|
| 1203 |
+
stamped = bridge.latest_matrix_stamped(side)
|
| 1204 |
+
if stamped is None:
|
| 1205 |
+
continue
|
| 1206 |
+
capture_ns, matrix = stamped
|
| 1207 |
+
try:
|
| 1208 |
+
encoded = encode_gripper_force_matrix(matrix, spec)
|
| 1209 |
+
except Exception as exc:
|
| 1210 |
+
self._log(tr("[夹爪] 力矩阵编码失败 ({}): {}",
|
| 1211 |
+
side, exc))
|
| 1212 |
+
continue
|
| 1213 |
+
self._pipeline.write_tactile_force_matrix(
|
| 1214 |
+
side, encoded, prefix=entry["snapshot_prefix"],
|
| 1215 |
+
capture_ns=capture_ns)
|
| 1216 |
+
wrote[side] += 1
|
| 1217 |
+
if not first_logged[side]:
|
| 1218 |
+
first_logged[side] = True
|
| 1219 |
+
self._log(tr("[夹爪] {} 侧力矩阵首帧落盘 ({} 元素)",
|
| 1220 |
+
"左" if side == "left" else "右",
|
| 1221 |
+
len(encoded)))
|
| 1222 |
+
if (time.monotonic() - started >= 10.0
|
| 1223 |
+
and not first_logged["left"]
|
| 1224 |
+
and not first_logged["right"]):
|
| 1225 |
+
self._log(tr(
|
| 1226 |
+
"[夹爪] 力矩阵泵 10s 无新样本 (L seq={} R seq={})",
|
| 1227 |
+
bridge.matrix_seq("left"), bridge.matrix_seq("right")))
|
| 1228 |
+
started = time.monotonic()
|
| 1229 |
+
stop.wait(0.03)
|
| 1230 |
+
|
| 1231 |
+
def _stop_matrix_pump(self, entry):
|
| 1232 |
+
"""录制结束/设备关闭:停泵线程并等待退出。"""
|
| 1233 |
+
stop = entry.get("matrix_pump_stop")
|
| 1234 |
+
if stop is not None:
|
| 1235 |
+
stop.set()
|
| 1236 |
+
entry["matrix_pump_stop"] = None
|
| 1237 |
+
thread = entry.pop("matrix_pump_thread", None)
|
| 1238 |
+
if thread is not None and thread.is_alive():
|
| 1239 |
+
thread.join(timeout=1.0)
|
| 1240 |
+
wrote = entry.pop("matrix_wrote", None)
|
| 1241 |
+
if wrote is not None:
|
| 1242 |
+
self._log(tr("[夹爪] 力矩阵泵停止:L 写 {} 帧 R 写 {} 帧",
|
| 1243 |
+
wrote["left"], wrote["right"]))
|
| 1244 |
+
|
| 1245 |
# ── 录制控制 ──────────────────────────────────────
|
| 1246 |
|
| 1247 |
def _device_meta(self) -> List[dict]:
|
|
|
|
| 1255 |
meta.append({"key": key, "kind": "uvc", "name": e["label"],
|
| 1256 |
"serial": e.get("serial", ""),
|
| 1257 |
"slots": e["slots"]})
|
| 1258 |
+
elif e.get("kind") == "gripper":
|
| 1259 |
+
# 夹爪条目没有 "role" 键,必须显式分支(否则下面 else
|
| 1260 |
+
# 取 e["role"] 直接 KeyError)
|
| 1261 |
+
meta.append({"key": key, "kind": "gripper", "name": e["label"],
|
| 1262 |
+
"serial": e.get("serial", ""),
|
| 1263 |
+
"slots": e["slots"]})
|
| 1264 |
else:
|
| 1265 |
meta.append({"key": key, "kind": e["kind"],
|
| 1266 |
"name": e["label"], "serial": "",
|
| 1267 |
"slots": [e["role"]]})
|
| 1268 |
return meta
|
| 1269 |
|
| 1270 |
+
def _master_video_slot(self) -> Optional[str]:
|
| 1271 |
+
"""本次录制的主视频源槽,没有可用主源时返回 None。
|
| 1272 |
+
|
| 1273 |
+
优先级 D435 → UVC → 夹爪 RGB。夹爪的 RGB 槽要到首帧才惰性注册
|
| 1274 |
+
(bridge 启动要几秒),所以「开了夹爪但还没出图」也算没有主源。
|
| 1275 |
+
"""
|
| 1276 |
+
if self._d435_key:
|
| 1277 |
+
return settings.D435_SLOT_RGB
|
| 1278 |
+
if self._uvc_key:
|
| 1279 |
+
return LITE_UVC_SLOT
|
| 1280 |
+
for entry in self._gripper_entries():
|
| 1281 |
+
rgb = entry["slot_map"]["rgb"]
|
| 1282 |
+
if rgb in entry["_video_registered"]:
|
| 1283 |
+
return rgb
|
| 1284 |
+
return None
|
| 1285 |
+
|
| 1286 |
+
def _reset_gripper_record_state(self):
|
| 1287 |
+
"""录制开始:重置夹爪双目 30fps 空桶看门狗 + 清空录制前累积的轨迹
|
| 1288 |
+
(口径同 main_window._record_all;parquet 的 slam_trajectory 列由
|
| 1289 |
+
writer 录制期独立累积,不受影响)。lite 无轨迹显示,清它只为让
|
| 1290 |
+
「本次录制空桶」统计与主程序同口径。"""
|
| 1291 |
+
for entry in self._gripper_entries():
|
| 1292 |
+
bridge = entry.get("bridge")
|
| 1293 |
+
if bridge is not None:
|
| 1294 |
+
bridge.reset_stereo_drop_watch()
|
| 1295 |
+
bridge.clear_trajectory()
|
| 1296 |
+
|
| 1297 |
def _start_recording(self):
|
| 1298 |
if self._pipeline.is_recording:
|
| 1299 |
return
|
| 1300 |
+
# pipeline.py:474 无源守卫是**静默 return**(点了开始没反应),
|
| 1301 |
+
# 故在前置检查里提示清楚
|
| 1302 |
+
master = self._master_video_slot()
|
| 1303 |
+
if master is None:
|
| 1304 |
+
if self._gripper_entries():
|
| 1305 |
+
QMessageBox.warning(
|
| 1306 |
+
self, tr("无法录制"),
|
| 1307 |
+
tr("夹爪视频尚未就绪(相机链路正在启动)。\n"
|
| 1308 |
+
"请稍候几秒再点「开始」,或先开启 D435 / UVC 摄像头。"))
|
| 1309 |
+
else:
|
| 1310 |
+
QMessageBox.warning(
|
| 1311 |
+
self, tr("无法录制"),
|
| 1312 |
+
tr("请先开启 D435 深度相机、UVC 摄像头或 UMI 夹爪。\n"
|
| 1313 |
+
"录制必须有视频源。"))
|
| 1314 |
return
|
| 1315 |
+
self._reset_gripper_record_state()
|
|
|
|
| 1316 |
self._pipeline.start_recording(
|
| 1317 |
master,
|
| 1318 |
task_name=self._task_edit.text().strip(),
|
|
|
|
| 1328 |
self._pipeline.abort_recording("")
|
| 1329 |
|
| 1330 |
def _on_recording_started(self, slot_id: str):
|
| 1331 |
+
# 力矩阵泵最先启动:后续任何 UI 锁调用即使异常也不影响落盘
|
| 1332 |
+
# (夹爪在场时启动泵线程,录制期才落 force_matrix 列)
|
| 1333 |
+
self._start_matrix_pump()
|
| 1334 |
self._rec_status.setText(tr("⏺ 录制中"))
|
| 1335 |
self._btn_start.setEnabled(False)
|
| 1336 |
self._btn_stop.setEnabled(True)
|
|
|
|
| 1338 |
self._set_device_buttons_enabled(False)
|
| 1339 |
|
| 1340 |
def _on_recording_finished(self, slot_id: str, session_path: str):
|
| 1341 |
+
# 空桶统计要在下一次录制重置看门狗之前取;停泵必须在写线程收尾前
|
| 1342 |
+
# (口径同 main_window._stop_all / _on_recording_finished)
|
| 1343 |
+
self._gripper_drop_summary()
|
| 1344 |
+
for entry in self._gripper_entries():
|
| 1345 |
+
self._stop_matrix_pump(entry)
|
| 1346 |
self._rec_status.setText(tr("待机"))
|
| 1347 |
self._btn_start.setEnabled(True)
|
| 1348 |
self._btn_stop.setEnabled(False)
|
|
|
|
| 1375 |
os.path.basename(session_path)))
|
| 1376 |
|
| 1377 |
def _on_recording_aborted(self, slot_id: str):
|
| 1378 |
+
# 中止走独立信号 recording_aborted,不经过 _on_recording_finished
|
| 1379 |
+
# ——必须单独停泵,否则泵线程活到下段录制(主程序同口径)
|
| 1380 |
+
for entry in self._gripper_entries():
|
| 1381 |
+
self._stop_matrix_pump(entry)
|
| 1382 |
self._rec_status.setText(tr("待机"))
|
| 1383 |
self._btn_start.setEnabled(True)
|
| 1384 |
self._btn_stop.setEnabled(False)
|
|
|
|
| 1508 |
|
| 1509 |
def closeEvent(self, event):
|
| 1510 |
self._scan_timer.stop()
|
| 1511 |
+
# gripper 必须在列:漏了会让 SLAM/相机服务进程在关窗后残留、占住双目
|
| 1512 |
+
for kind in ("d435", "uvc", "gripper", "usb", "ble"):
|
| 1513 |
for i, row in enumerate(self._dev_groups[kind]):
|
| 1514 |
if getattr(self, row["attr"]):
|
| 1515 |
try:
|
collector/ui/main_window.py
CHANGED
|
@@ -115,74 +115,11 @@ except ImportError:
|
|
| 115 |
_GLOVE_AVAILABLE = False
|
| 116 |
|
| 117 |
|
| 118 |
-
|
| 119 |
-
|
| 120 |
-
|
| 121 |
-
|
| 122 |
-
|
| 123 |
-
**注意**:省略 spec 时走的是函数签名默认 "int16"(兼容历史调用的冻结
|
| 124 |
-
契约),**不跟随工具栏的默认档**;生产路径(P4 泵)一律显式传入录制
|
| 125 |
-
开始时锁定的 spec,绝不省略。
|
| 126 |
-
|
| 127 |
-
"int16"(既有契约):不缩放,与
|
| 128 |
-
online/recording/lerobot_v3.quantize_force_matrix 一致的有符号补码行
|
| 129 |
-
差分(mod 2^16 可逆),扩展到 (250,250,3) 三力平面:每行 250×3 个
|
| 130 |
-
元素横向差分;超出 ±32767 mN 饱和截断。**逐点 astype(int16) 是向零
|
| 131 |
-
截断,小数部分全部丢失**——实测真实录制里逐点值只剩 -3…8 的整数电平,
|
| 132 |
-
亚毫牛梯度(接触形状)不可恢复。保留向零截断是为了与历史录制逐位一致。
|
| 133 |
-
|
| 134 |
-
"int16xN"(N=10/100/1000,定标量化):先 ×N 再四舍五入到整数存 int16,
|
| 135 |
-
解码端除以 N 还原,分辨率 1/N mN。饱和上限相应变为 ±32767/N mN
|
| 136 |
-
(×1000 即 ±32.767 mN)。定标档用 rint 而非截断:没有历史包袱,且
|
| 137 |
-
四舍五入把最大误差减半、不留系统性偏零。行差分与 int16 档完全相同,
|
| 138 |
-
所以仍然压得很小(×10 实测 0.122 字节/点,float32 的 1/6.5)。
|
| 139 |
-
|
| 140 |
-
"float32"(原值直存):不做任何量化,把 (250,250,3) 按行展平成 250×750
|
| 141 |
-
的 float32 列表。无截断、无饱和,误差为 float32 本身的 ~1e-7 相对精度。
|
| 142 |
-
体积换精度:实测 0.797 字节/点(本段数据约为 int16 档的 119 倍)。
|
| 143 |
-
|
| 144 |
-
反解:整数档 reshape(-1,750) → cumsum(axis=1) → reshape(250,250,3)
|
| 145 |
-
(cumsum 走 int32 再截断回 int16 恢复 mod 2^16 回绕),
|
| 146 |
-
再按倍率 ÷N;float32 档 reshape(-1,750) → (250,250,3)。
|
| 147 |
-
|
| 148 |
-
★ 返回的 Python 元素类型只区分「家族」:int = int16 系(**倍率无法从
|
| 149 |
-
元素类型看出**,×10 和 ×1000 都是 int),float = float32。
|
| 150 |
-
倍率必须由 info.json features.scale 携带,writer 与 demo 都按
|
| 151 |
-
scale + 列类型共同判别,不要只看元素类型。
|
| 152 |
-
"""
|
| 153 |
-
if spec not in settings.GRIPPER_FORCE_MATRIX_SPECS:
|
| 154 |
-
raise ValueError(f"未知力矩阵规格 {spec!r}")
|
| 155 |
-
arr = np.asarray(m, dtype=np.float32)
|
| 156 |
-
if spec == "float32":
|
| 157 |
-
# 与整数档同为「扁平 250×750 列表」,只有元素类型不同:
|
| 158 |
-
# 读取端统一 reshape(-1,750) 后再按 dtype 决定是否 cumsum
|
| 159 |
-
return arr.reshape(-1).tolist()
|
| 160 |
-
scale = settings.GRIPPER_FORCE_MATRIX_SCALES[spec]
|
| 161 |
-
rows = int(arr.shape[0])
|
| 162 |
-
flat = arr.reshape(rows, -1)
|
| 163 |
-
if scale == 1:
|
| 164 |
-
q = np.clip(flat, -32767.0, 32767.0).astype(np.int16)
|
| 165 |
-
else:
|
| 166 |
-
# rint 后已是整数值,astype 的向零截断不再丢任何东西
|
| 167 |
-
q = np.clip(np.rint(flat * np.float32(scale)),
|
| 168 |
-
-32767.0, 32767.0).astype(np.int16)
|
| 169 |
-
d = np.empty_like(q)
|
| 170 |
-
d[:, 0] = q[:, 0]
|
| 171 |
-
d[:, 1:] = q[:, 1:] - q[:, :-1] # int16 mod 2^16 回绕
|
| 172 |
-
return np.frombuffer(d.tobytes(), dtype=np.int16).tolist()
|
| 173 |
-
|
| 174 |
-
|
| 175 |
-
def describe_gripper_matrix_spec(spec: str) -> str:
|
| 176 |
-
"""规格 → 人话(工具栏提示与日志共用,别在两处各写一份)。"""
|
| 177 |
-
if spec not in settings.GRIPPER_FORCE_MATRIX_SPECS:
|
| 178 |
-
return tr("未知规格 {}", spec)
|
| 179 |
-
scale = settings.GRIPPER_FORCE_MATRIX_SCALES.get(spec)
|
| 180 |
-
if scale is None:
|
| 181 |
-
return tr("float32 原值直存(无截断无饱和,体积最大)")
|
| 182 |
-
if scale == 1:
|
| 183 |
-
return tr("int16 行差分量化(1 mN 台阶、向零截断,体积最小)")
|
| 184 |
-
return tr("int16 定标 ×{}(分辨率 {} mN,饱和上限 ±{} mN)",
|
| 185 |
-
scale, 1.0 / scale, 32767.0 / scale)
|
| 186 |
|
| 187 |
|
| 188 |
class MainWindow(QMainWindow):
|
|
@@ -1661,9 +1598,10 @@ class MainWindow(QMainWindow):
|
|
| 1661 |
self._on_gripper_tactile(key, side, heatmap, force, matrix,
|
| 1662 |
capture_ns))
|
| 1663 |
bridge.pose_ready.connect(
|
| 1664 |
-
lambda position, quat, trajectory=(), timestamp=None,
|
|
|
|
| 1665 |
self._on_gripper_pose(key, position, quat, trajectory,
|
| 1666 |
-
timestamp))
|
| 1667 |
bridge.gripper_state_ready.connect(
|
| 1668 |
lambda payload, key=key: self._on_gripper_state(key, payload))
|
| 1669 |
bridge.log.connect(self._log)
|
|
@@ -1797,11 +1735,13 @@ class MainWindow(QMainWindow):
|
|
| 1797 |
widget.video_widget.set_frame(frame, flip_vertical=False)
|
| 1798 |
|
| 1799 |
def _on_gripper_pose(self, dev_key: str, position, quat, trajectory=(),
|
| 1800 |
-
timestamp=None):
|
| 1801 |
-
"""SLAM 位姿(pos3, quat4, traj, t):quat→3x3 驱动
|
| 1802 |
-
(含轨迹) + 轨迹点落盘(rig2 加前缀);t
|
| 1803 |
-
slam_trajectory 列(轨迹并入
|
| 1804 |
-
位姿本身不再落 slam_pose 列
|
|
|
|
|
|
|
| 1805 |
entry = self._gripper_entry(dev_key)
|
| 1806 |
if entry is None:
|
| 1807 |
return
|
|
@@ -1824,7 +1764,8 @@ class MainWindow(QMainWindow):
|
|
| 1824 |
self._pipeline.write_slam_trajectory(
|
| 1825 |
[float(position[0]), float(position[1]), float(position[2]),
|
| 1826 |
qx, qy, qz, qw],
|
| 1827 |
-
prefix=entry["snapshot_prefix"], timestamp=timestamp
|
|
|
|
| 1828 |
|
| 1829 |
def _on_gripper_tactile(self, dev_key: str, side, heatmap, force,
|
| 1830 |
force_matrix, capture_ns=0):
|
|
|
|
| 115 |
_GLOVE_AVAILABLE = False
|
| 116 |
|
| 117 |
|
| 118 |
+
# 力矩阵编码与规格文案是**落盘契约**(家在 core/gripper_codec.py —— 极简版
|
| 119 |
+
# lite 也要录夹爪,而 lite 的导入黑名单不含本模块,契约不能存两份)。
|
| 120 |
+
# 这里 re-export,本窗口与 tools/tests/test_gripper_matrix_*.py 沿用旧路径。
|
| 121 |
+
from core.gripper_codec import ( # noqa: E402,F401
|
| 122 |
+
encode_gripper_force_matrix, describe_gripper_matrix_spec)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 123 |
|
| 124 |
|
| 125 |
class MainWindow(QMainWindow):
|
|
|
|
| 1598 |
self._on_gripper_tactile(key, side, heatmap, force, matrix,
|
| 1599 |
capture_ns))
|
| 1600 |
bridge.pose_ready.connect(
|
| 1601 |
+
lambda position, quat, trajectory=(), timestamp=None,
|
| 1602 |
+
host_ns=None, key=key:
|
| 1603 |
self._on_gripper_pose(key, position, quat, trajectory,
|
| 1604 |
+
timestamp, host_ns))
|
| 1605 |
bridge.gripper_state_ready.connect(
|
| 1606 |
lambda payload, key=key: self._on_gripper_state(key, payload))
|
| 1607 |
bridge.log.connect(self._log)
|
|
|
|
| 1735 |
widget.video_widget.set_frame(frame, flip_vertical=False)
|
| 1736 |
|
| 1737 |
def _on_gripper_pose(self, dev_key: str, position, quat, trajectory=(),
|
| 1738 |
+
timestamp=None, host_ns=None):
|
| 1739 |
+
"""SLAM 位姿(pos3, quat4, traj, t, host_ns):quat→3x3 驱动
|
| 1740 |
+
PoseViewQt(含轨迹) + 轨迹点落盘(rig2 加前缀);t 与 host_ns
|
| 1741 |
+
随位姿进入 slam_trajectory / slam_trajectory_ns 列(轨迹并入
|
| 1742 |
+
episode parquet,无 txt 侧车)。位姿本身不再落 slam_pose 列
|
| 1743 |
+
(2026-09-10 定案)。host_ns = 该取样帧的宿主单调钟纳秒
|
| 1744 |
+
(与 hardware_ns 同时基),是 slam 点与视频帧按时间对齐的依据。"""
|
| 1745 |
entry = self._gripper_entry(dev_key)
|
| 1746 |
if entry is None:
|
| 1747 |
return
|
|
|
|
| 1764 |
self._pipeline.write_slam_trajectory(
|
| 1765 |
[float(position[0]), float(position[1]), float(position[2]),
|
| 1766 |
qx, qy, qz, qw],
|
| 1767 |
+
prefix=entry["snapshot_prefix"], timestamp=timestamp,
|
| 1768 |
+
host_ns=host_ns)
|
| 1769 |
|
| 1770 |
def _on_gripper_tactile(self, dev_key: str, side, heatmap, force,
|
| 1771 |
force_matrix, capture_ns=0):
|
collector/使用手册.md
CHANGED
|
@@ -89,7 +89,8 @@ After extraction you will see these items; only the listed ones matter to you:
|
|
| 89 |
| `run.bat` / `run.sh` | Direct launch (fastest, for everyday use after setup) |
|
| 90 |
| `main.py` | Main program entry (you won't run it manually) |
|
| 91 |
| `data/` | **All your recordings + local config live here** — back up by copying the whole folder |
|
| 92 |
-
| `
|
|
|
|
| 93 |
| `wheels/` | Offline installer packages (prepared by admins; ignore if absent) |
|
| 94 |
| `docs/` | Documentation set |
|
| 95 |
|
|
@@ -200,14 +201,16 @@ The Lite edition is a stripped-down collector in its own folder with its own env
|
|
| 200 |
|
| 201 |
| Step | Action |
|
| 202 |
|---|---|
|
| 203 |
-
|
|
| 204 |
| Double-click | `start_lite.bat` (Windows) / `./start_lite.sh` (Linux) |
|
| 205 |
| Automatic | Same six steps → independent venv `venv_lite/` → 9 slim dependencies → self-check → launch |
|
| 206 |
-
| Size | ~750 MB environment, smaller than the main edition |
|
|
|
|
| 207 |
|
| 208 |

|
| 209 |
|
| 210 |
- Sub-commands: `reinstall`, `help`; offline: `pack_wheels.py --lite`.
|
|
|
|
| 211 |
- **Data is shared**: Lite and main editions use the same `data/` folder — recordings made in Lite can be played back and uploaded in the main edition (see Appendix C).
|
| 212 |
|
| 213 |
<a id="en-ch2-8"></a>
|
|
@@ -322,7 +325,7 @@ Note: the choice is **not saved** — the program restarts in Chinese. Switch it
|
|
| 322 |
The Lite edition does exactly three things: **connect devices → record → upload**. No login, no task page, no playback. Window 980×640, title "DAQ 极简采集 (lite v1.2.0)". Top to bottom:
|
| 323 |
|
| 324 |
1. **Server row**: address field + "Test Connection" button + "Auto Upload" and "Delete After Upload" checkboxes;
|
| 325 |
-
2. **Left device area**:
|
| 326 |
3. **Right preview**: 640×360 feed at 15 fps; "Preview Source" dropdown switches RGB / Depth Heatmap / UVC RGB;
|
| 327 |
4. **Recording row**: task-name field (optional) + "▶ Start" (green) "■ Stop" (red) "⛔ Discard" (orange) + status text (Standby / ⏺ Recording HH:MM:SS);
|
| 328 |
5. **Upload row**: pending list ("task —— status (ep n)": Not Uploaded / Uploading / Uploaded / Failed) + "Upload Selected" / "Upload All Pending" + progress bar;
|
|
@@ -445,6 +448,7 @@ One table, everything you can plug in. **The universal flow: plug in → wait ~2
|
|
| 445 |
| Stereo S80C / S80M | USB 3.0 | 1 | Capture 50 fps → recorded at 30 fps | SDK built in; cannot run together with RealSense (6.3) |
|
| 446 |
| USB Type-C glove | USB serial (STM32 CDC) | 1 per hand | 16 IMU sensors + 16×16 tactile, rated 60 fps | No driver; 115200 baud |
|
| 447 |
| BLE glove | Bluetooth | 1 per hand | 16×16 tactile | PC Bluetooth on; broadcast name L/R picks the hand |
|
|
|
|
| 448 |
|
| 449 |

|
| 450 |
|
|
@@ -670,7 +674,7 @@ Depth has its own **anti-drop mechanism** (repeats the last frame when one is mi
|
|
| 670 |
|
| 671 |
Three buttons, one straight flow:
|
| 672 |
|
| 673 |
-
1. Open at least one device first — **D435 or
|
| 674 |
2. Task name field is optional (recordings fall under the default task);
|
| 675 |
3. **▶ Start** → status shows "⏺ Recording HH:MM:SS"; **■ Stop** = save; **⛔ Discard** = throw this take away;
|
| 676 |
4. Fixed Lite parameters: UVC records 640×480@30, preview 640×360@15 fps; **encoder fixed at H.264/x264** (no probe, low-end friendly);
|
|
@@ -818,6 +822,7 @@ data/recordings/<task name>/
|
|
| 818 |
| `d435_rgb` / `D435_head_depth` | RealSense D435 color / depth |
|
| 819 |
| `stereo_left` / `stereo_right` | Stereo camera left / right |
|
| 820 |
| `stereo_depth` | Stereo disparity depth |
|
|
|
|
| 821 |
|
| 822 |
<a id="en-ch9-3"></a>
|
| 823 |
|
|
@@ -847,6 +852,13 @@ data/recordings/<task name>/
|
|
| 847 |
| `observation.<sn>_imu_quat` | USB glove IMU: 16 sensors × 4-component quaternion = 64 dims |
|
| 848 |
| `observation.<sn>_imu_valid` | Validity flags for those 16 sensors (16 dims) |
|
| 849 |
| `observation.left_hand_pose` / `observation.right_hand_pose` | Hand keypoints: **63 dims = 21 points × 3 coordinates (meters)**; zeros when not solved (placeholder column) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 850 |
|
| 851 |
**③ Action & status**
|
| 852 |
|
|
@@ -924,7 +936,7 @@ Besides recordings, `data/` holds a few small program files:
|
|
| 924 |
| 6-2 | 6.1 | One UVC tile (info bar: name/FPS/light) | `docs/images/fig-6-2.png` | Main | 800×450 |
|
| 925 |
| 6-3 | 6.2 | D435 two tiles (RGB + depth heatmap) | `docs/images/fig-6-3.png` | Main | 1200×450 |
|
| 926 |
| 6-4 | 6.3 | Stereo three tiles (left/right/depth) | `docs/images/fig-6-4.png` | Main | 1400×450 |
|
| 927 |
-
| 6-6 | 6.0 | Lite device area close-up (
|
| 928 |
| 7-1 | 7.1/7.2 | Recording in progress (⏱ timer, "Recording: 1", locked panel) | `docs/images/fig-7-1.png` | Main | 1600×900 |
|
| 929 |
| 7-2 | 7.1 | Progress-full refusal dialog | `docs/images/fig-7-2.png` | Main | native |
|
| 930 |
| 7-3 | 7.4 | Leave-confirmation dialog while recording | `docs/images/fig-7-3.png` | Main | native |
|
|
@@ -954,6 +966,7 @@ Besides recordings, `data/` holds a few small program files:
|
|
| 954 |
| Code | Symptom | Solution |
|
| 955 |
|---|---|---|
|
| 956 |
| **A** | Python not found / auto-install failed | ① Offline: put `python-3.12.10-amd64.exe` into `wheels/` and retry; ② Install Python 3.12.x 64-bit manually, tick **Add python.exe to PATH**; ③ A Microsoft Store Python may interfere — uninstall it, install the official one; ④ Python < 3.10 counts as missing |
|
|
|
|
| 957 |
| **C** | venv creation failed | ① Free disk space (needs ~2 GB); ② Antivirus blocked it → whitelist the folder, then `start.bat reinstall`; ③ Path too long → move to a short path like `C:\collector`; ④ Special characters in path → use a plain-English folder |
|
| 958 |
| **C2** | Old venv deletion failed | The main program is still running — close its window, then `start.bat reinstall` |
|
| 959 |
| **D** | Dependency download/install failed | ① Check the network and rerun (partial downloads are cached); ② Corporate firewall → ask admin to allow PyPI mirrors or use the offline route (2.6); ③ Antivirus blocking pip → whitelist; ④ Repeated failures → `start.bat reinstall` |
|
|
@@ -1015,7 +1028,9 @@ Not sure which edition to use? This table settles it:
|
|
| 1015 |
| Quick-start guide | Yes (pops on first launch) | No |
|
| 1016 |
| Hand skeleton solving | Yes (MANO 21 keypoints filled live) | No (columns stay zero — valid) |
|
| 1017 |
| Theme | qt-material dark theme | Lightweight dark style |
|
| 1018 |
-
| Devices | UVC ≤ 8 + D435 + stereo S80C/S80M + gloves | One of each: D435 / UVC / USB-C glove; BLE gloves 2 (left + right at once) |
|
|
|
|
|
|
|
| 1019 |
| UVC recording | Linux 1280×960, Windows 1280×720 @30 | Fixed 640×480 @30 |
|
| 1020 |
| Preview | Full res, capped 30 fps | 640×360 @15 fps |
|
| 1021 |
| Encoder | Auto probe (nvenc → x265 → x264) | Fixed x264 (low-end friendly) |
|
|
@@ -1077,7 +1092,7 @@ Real-time 3D hand-keypoint demo for the S80C stereo camera (`run_live_s80c.sh`,
|
|
| 1077 |
| 使用说明.md / 使用说明_EN.md | Quick guide + error-code reference (opened by `start.bat help`) |
|
| 1078 |
| README.md | Repo overview & dev info (hardware table, changelog) |
|
| 1079 |
| docs/ (ui.md, data.md, file_format.md …) | Implementation details, advanced reading |
|
| 1080 |
-
|
|
| 1081 |
| tools/glove_package/使用说明.md | Glove toolchain details |
|
| 1082 |
|
| 1083 |
**When reporting a problem**, include these three things to speed things up:
|
|
@@ -1182,7 +1197,8 @@ Real-time 3D hand-keypoint demo for the S80C stereo camera (`run_live_s80c.sh`,
|
|
| 1182 |
| `run.bat` / `run.sh` | 直接启动(部署完成后日常用它,更快) |
|
| 1183 |
| `main.py` | 主程序入口(不需要手动运行) |
|
| 1184 |
| `data/` | **你录的全部数据 + 本地配置都在这里**,备份就拷贝整个目录 |
|
| 1185 |
-
| `
|
|
|
|
| 1186 |
| `wheels/` | 离线安装包(管理员准备的,普通用户没有就忽略) |
|
| 1187 |
| `docs/` | 文档集(开发文档,进阶阅读用) |
|
| 1188 |
|
|
@@ -1303,14 +1319,16 @@ python scripts/pack_wheels.py --lite # 极简版依赖
|
|
| 1303 |
|
| 1304 |
| 步骤 | 操作 |
|
| 1305 |
|---|---|
|
| 1306 |
-
|
|
| 1307 |
| 双击 | `start_lite.bat`(Windows)/ `./start_lite.sh`(Linux) |
|
| 1308 |
| 自动完成 | 同样六步:定位 Python → 建独立虚拟环境 `venv_lite/` → 装精简依赖(9 个包)→ 自检 → 启动 |
|
| 1309 |
-
| 体积 | 环境约 750MB,比主程序小 |
|
|
|
|
| 1310 |
|
| 1311 |

|
| 1312 |
|
| 1313 |
- 子命令同样支持:`reinstall`(重装)、`help`(打开极简版说明)。
|
|
|
|
| 1314 |
- 离线环境用 `python scripts/pack_wheels.py --lite` 生成 `wheels/` 后同法拷贝。
|
| 1315 |
- **数据互通**:极简版与主程序共用 `data/` 目录——极简版录��数据,主程序照样能回放、上传(见附录 C)。
|
| 1316 |
|
|
@@ -1426,7 +1444,7 @@ python scripts/pack_wheels.py --lite # 极简版依赖
|
|
| 1426 |
极简版定位「**连设备 → 采集 → 上传**」三件事,砍掉了登录、任务页、回放等一切多余功能。窗口 980×640,标题「DAQ 极简采集 (lite v1.2.0)」,自上而下:
|
| 1427 |
|
| 1428 |
1. **顶部服务器行**:服务器地址输入框 + 「测试连接」按钮 + 「自动上传」「上传后删除」两个勾选框;
|
| 1429 |
-
2. **左侧设备区**:
|
| 1430 |
3. **右侧预览区**:640×360 实时画面(15fps),「预览源」下拉可切换 RGB / 深度热力图 / UVC RGB;
|
| 1431 |
4. **录制行**:任务名输入框(可选填)+ 「▶ 开始」(绿)「■ 停止」(红)「⛔ 丢弃」(橙)三个按钮 + 状态文字(待机 / ⏺ 录制中 时长);
|
| 1432 |
5. **上传行**:待上传列表(显示 任务名 —— 状态 (ep n),状态有 未上传/上传中/已上传/失败)+「上传选中」「上传全部未上传」按钮 + 上传进度条;
|
|
@@ -1553,6 +1571,7 @@ python scripts/pack_wheels.py --lite # 极简版依赖
|
|
| 1553 |
| 双目相机 S80C / S80M | USB 3.0 | 1 | 采集 50fps → 录制抽帧 30fps | 免装 SDK(程序自带);与 RealSense 不能同开(见 6.3) |
|
| 1554 |
| USB Type-C 手套 | USB 串口(STM32 CDC) | 左右各 1 | 16 传感器 IMU + 16×16 触觉,额定 60fps | 免驱动;串口波特率 115200 |
|
| 1555 |
| BLE 蓝牙手套 | 蓝牙 | 左右各 1 | 16×16 触觉 | 电脑需开蓝牙;广播名含 L/R 自动分左右手 |
|
|
|
|
| 1556 |
|
| 1557 |

|
| 1558 |
|
|
@@ -1782,7 +1801,7 @@ RECORD_VIDEO_ENCODER = "auto" # 自动(默认);也可填 "nvenc" | "x265
|
|
| 1782 |
|
| 1783 |
极简版的录制只有三个按钮,流程更直白:
|
| 1784 |
|
| 1785 |
-
1. 先在左侧设备区开启至少一台设备——**必须先开 D435 或
|
| 1786 |
2. 任务名输入框选填(不填也能录,数据归到默认任务下);
|
| 1787 |
3. 点 **「▶ 开始」** → 录制中显示「⏺ 录制中 时长」;点 **「■ 停止」** = 保存;点 **「⛔ 丢弃」** = 作废本段;
|
| 1788 |
4. 极简版的固定参数:UVC 录制 640×480@30、预览 640×360@15fps;**编码固定 H.264/x264**(跳过编码器探针,低端机友好);
|
|
@@ -1935,6 +1954,7 @@ data/recordings/<任务名>/
|
|
| 1935 |
| `d435_rgb` / `D435_head_depth` | RealSense D435 的彩色 / 深度 |
|
| 1936 |
| `stereo_left` / `stereo_right` | 双目相机左 / 右目 |
|
| 1937 |
| `stereo_depth` | 双目视差深度 |
|
|
|
|
| 1938 |
|
| 1939 |
<a id="cn-ch9-3"></a>
|
| 1940 |
|
|
@@ -1964,6 +1984,13 @@ data/recordings/<任务名>/
|
|
| 1964 |
| `observation.<sn>_imu_quat` | USB 手套 IMU:16 传感器 × 4 分量四元数 = 64 维 |
|
| 1965 |
| `observation.<sn>_imu_valid` | 对应 16 个传感器有效性标志(16 维) |
|
| 1966 |
| `observation.left_hand_pose` / `observation.right_hand_pose` | 手部关键点:**63 维 = 21 点 × 3 坐标(米)**;未解算时是 0 占位列 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1967 |
|
| 1968 |
**③ 动作与状态**
|
| 1969 |
|
|
@@ -2041,7 +2068,7 @@ data/recordings/<任务名>/
|
|
| 2041 |
| 6-2 | 6.1 UVC | 一路 UVC 画面格(信息条:设备名/FPS/状态灯) | `docs/images/fig-6-2.png` | 普通版 | 800×450 |
|
| 2042 |
| 6-3 | 6.2 RealSense | D435 两格画面(RGB + 深度热力图) | `docs/images/fig-6-3.png` | 普通版 | 1200×450 |
|
| 2043 |
| 6-4 | 6.3 双目 | S80C 三格画面(左目/右目/深度热力图) | `docs/images/fig-6-4.png` | 普通版 | 1400×450 |
|
| 2044 |
-
| 6-6 | 6.0 极简版设备区 | 极简版左列设备区特写(
|
| 2045 |
| 7-1 | 7.1/7.2 录制中 | 录制中整窗(⏱计时、录制中: 1、面板锁定态) | `docs/images/fig-7-1.png` | 普通版 | 1600×900 |
|
| 2046 |
| 7-2 | 7.1 拒绝录制 | 「任务进度已满」拒绝弹窗 | `docs/images/fig-7-2.png` | 普通版 | 对话框原生 |
|
| 2047 |
| 7-3 | 7.4 中断确认 | 录制中返回任务选择的确认弹窗 | `docs/images/fig-7-3.png` | 普通版 | 对话框原生 |
|
|
@@ -2071,6 +2098,7 @@ data/recordings/<任务名>/
|
|
| 2071 |
| 错误码 | 现象 | 解决方案 |
|
| 2072 |
|---|---|---|
|
| 2073 |
| **A** | 找不到 Python / 自动安装失败 | ① 离线环境:把 `python-3.12.10-amd64.exe` 放入 `wheels/` 后重试;② 手动安装:官网下载 Python 3.12.x **64 位**,安装时勾选 **Add python.exe to PATH**;③ 已装过 Python:可能是 Microsoft Store 版干扰,在「设置-应用」卸载后装官网版;④ 已有 Python 但版本低于 3.10 同样走 ② |
|
|
|
|
| 2074 |
| **C** | 虚拟环境创建失败 | ① 磁盘空间不足(需约 2GB);② 杀毒软件拦截 → 目录加白名单后 `start.bat reinstall`;③ 路径过长 → 移到 `C:\collector` 之类短路径;④ 路径含特殊字符 → 换纯英文目录 |
|
| 2075 |
| **C2** | 旧 venv 删除失败 | 主程序还在运行,先关闭窗口再 `start.bat reinstall` |
|
| 2076 |
| **D** | 依赖下载/安装失败 | ① 网络不通 → 检查网络后重跑(已下载部分有缓存,不重复);② 公司网络限制 → 请管理员放行 PyPI 镜像,或走离线交付(见第 2.6 节);③ 杀毒/防火墙拦 pip → 白名单;④ 多次失败 → `start.bat reinstall` |
|
|
@@ -2135,7 +2163,9 @@ data/recordings/<任务名>/
|
|
| 2135 |
| 使用说明向导 | 有(首次启动弹出) | 无 |
|
| 2136 |
| 手部骨架解算 | 有(MANO 21 关键点实时回填) | 无(数据列留 0 占位,合法) |
|
| 2137 |
| 界面主题 | qt-material 暗色主题 | 轻量暗色样式 |
|
| 2138 |
-
| 设备数量 | UVC ≤ 8 路 + D435 + 双目 S80C/S80M + 手套 | 每类 1 台:D435 / UVC / USB-C 手套;BLE 手套 2 台(左右手同时) |
|
|
|
|
|
|
|
| 2139 |
| UVC 录制分辨率 | Linux 1280×960、Windows 1280×720 @30 | 固定 640×480 @30 |
|
| 2140 |
| 预览 | 全分辨率,30fps 上限 | 640×360 @15fps |
|
| 2141 |
| 编码器 | 自动探测(nvenc → x265 → x264) | 固定 x264(低端机友好) |
|
|
@@ -2197,7 +2227,7 @@ S80C 双目相机的实时 3D 手部关键点 demo(`run_live_s80c.sh`,`--glo
|
|
| 2197 |
| 使用说明.md / 使用说明_EN.md | 快速指引 + 错误码速查(`start.bat help` 打开它) |
|
| 2198 |
| README.md | 仓库总览与开发信息(硬件支持表、更新记录) |
|
| 2199 |
| docs/(ui.md、data.md、file_format.md 等) | 模块与格式实现细节,进阶阅读 |
|
| 2200 |
-
|
|
| 2201 |
| tools/glove_package/使用说明.md | 手套识别工具链详细说明 |
|
| 2202 |
|
| 2203 |
**遇到问题反馈时**,请附上三样东西,能大幅加快排查:
|
|
|
|
| 89 |
| `run.bat` / `run.sh` | Direct launch (fastest, for everyday use after setup) |
|
| 90 |
| `main.py` | Main program entry (you won't run it manually) |
|
| 91 |
| `data/` | **All your recordings + local config live here** — back up by copying the whole folder |
|
| 92 |
+
| `start_lite.bat` / `start_lite.sh` | **Lite edition** one-click setup + launch (own env `venv_lite/`), see 2.7 |
|
| 93 |
+
| `lite_package/` | Lite edition distribution folder (a ready-to-ship copy of the same thing; on Linux it also carries the ~460 MB gripper payload — see 2.7) |
|
| 94 |
| `wheels/` | Offline installer packages (prepared by admins; ignore if absent) |
|
| 95 |
| `docs/` | Documentation set |
|
| 96 |
|
|
|
|
| 201 |
|
| 202 |
| Step | Action |
|
| 203 |
|---|---|
|
| 204 |
+
| Run in | the folder holding `main_lite.py` — in this repo that is the **root** (see 1.3); `lite_package/` is the packaged copy |
|
| 205 |
| Double-click | `start_lite.bat` (Windows) / `./start_lite.sh` (Linux) |
|
| 206 |
| Automatic | Same six steps → independent venv `venv_lite/` → 9 slim dependencies → self-check → launch |
|
| 207 |
+
| Size | ~750 MB environment, smaller than the main edition; the **Linux** package adds the gripper native stack (~460 MB payload → ~1.22 GB total), the Windows package does not |
|
| 208 |
+
| Gripper | **UMI gripper is supported on Linux only, and records without any visualization** (no stereo view, no force-matrix heatmap, no pose view — data is written to disk as usual). The Windows package ships the gripper code but no payload, so the "UMI Gripper" group stays empty — the group box is always there (like every other group), the dropdown just has no entries and clicking "Open" answers "no device in the list". Expected, not a fault |
|
| 209 |
|
| 210 |

|
| 211 |
|
| 212 |
- Sub-commands: `reinstall`, `help`; offline: `pack_wheels.py --lite`.
|
| 213 |
+
- The **Linux** launcher additionally runs a 7-item gripper resource check (`[B]`) and refuses to start with `[Error B]` if the payload is missing; `KSQ_GRIPPER_SKIP_CHECK=1 ./start_lite.sh` skips it (gripper then just doesn't show up).
|
| 214 |
- **Data is shared**: Lite and main editions use the same `data/` folder — recordings made in Lite can be played back and uploaded in the main edition (see Appendix C).
|
| 215 |
|
| 216 |
<a id="en-ch2-8"></a>
|
|
|
|
| 325 |
The Lite edition does exactly three things: **connect devices → record → upload**. No login, no task page, no playback. Window 980×640, title "DAQ 极简采集 (lite v1.2.0)". Top to bottom:
|
| 326 |
|
| 327 |
1. **Server row**: address field + "Test Connection" button + "Auto Upload" and "Delete After Upload" checkboxes;
|
| 328 |
+
2. **Left device area**: five groups (D435 Depth Camera / UVC Camera / USB-C Glove / BLE Glove / UMI Gripper, Linux package only), each a device dropdown + an "Open" button (becomes "Close"; the BLE Glove group has two rows, so you can connect the left and right gloves at the same time), plus a "Rescan" button;
|
| 329 |
3. **Right preview**: 640×360 feed at 15 fps; "Preview Source" dropdown switches RGB / Depth Heatmap / UVC RGB;
|
| 330 |
4. **Recording row**: task-name field (optional) + "▶ Start" (green) "■ Stop" (red) "⛔ Discard" (orange) + status text (Standby / ⏺ Recording HH:MM:SS);
|
| 331 |
5. **Upload row**: pending list ("task —— status (ep n)": Not Uploaded / Uploading / Uploaded / Failed) + "Upload Selected" / "Upload All Pending" + progress bar;
|
|
|
|
| 448 |
| Stereo S80C / S80M | USB 3.0 | 1 | Capture 50 fps → recorded at 30 fps | SDK built in; cannot run together with RealSense (6.3) |
|
| 449 |
| USB Type-C glove | USB serial (STM32 CDC) | 1 per hand | 16 IMU sensors + 16×16 tactile, rated 60 fps | No driver; 115200 baud |
|
| 450 |
| BLE glove | Bluetooth | 1 per hand | 16×16 tactile | PC Bluetooth on; broadcast name L/R picks the hand |
|
| 451 |
+
| UMI gripper (Fays S80M) | USB 3.0 + serial | 2 | Left-lens RGB + 10×10×3 force matrix + SLAM trajectory, 30 fps | **Linux only** (native stack is ELF); needs the 460 MB payload (see 2.7 and Appendix C) |
|
| 452 |
|
| 453 |

|
| 454 |
|
|
|
|
| 674 |
|
| 675 |
Three buttons, one straight flow:
|
| 676 |
|
| 677 |
+
1. Open at least one device first — **D435, UVC, or the UMI gripper (Linux) is mandatory**: gloves alone can't record (no video source; the program says "请先开启 D435 深度相机、UVC 摄像头或 UMI 夹爪。录制必须有视频源。"). A gripper-only setup records its left-lens RGB as the master video;
|
| 678 |
2. Task name field is optional (recordings fall under the default task);
|
| 679 |
3. **▶ Start** → status shows "⏺ Recording HH:MM:SS"; **■ Stop** = save; **⛔ Discard** = throw this take away;
|
| 680 |
4. Fixed Lite parameters: UVC records 640×480@30, preview 640×360@15 fps; **encoder fixed at H.264/x264** (no probe, low-end friendly);
|
|
|
|
| 822 |
| `d435_rgb` / `D435_head_depth` | RealSense D435 color / depth |
|
| 823 |
| `stereo_left` / `stereo_right` | Stereo camera left / right |
|
| 824 |
| `stereo_depth` | Stereo disparity depth |
|
| 825 |
+
| `gripper_rgb` (`gripper_2_rgb` for the 2nd gripper) | UMI gripper left lens (recorded whenever a gripper is open — in the Lite edition this is the only gripper video) |
|
| 826 |
|
| 827 |
<a id="en-ch9-3"></a>
|
| 828 |
|
|
|
|
| 852 |
| `observation.<sn>_imu_quat` | USB glove IMU: 16 sensors × 4-component quaternion = 64 dims |
|
| 853 |
| `observation.<sn>_imu_valid` | Validity flags for those 16 sensors (16 dims) |
|
| 854 |
| `observation.left_hand_pose` / `observation.right_hand_pose` | Hand keypoints: **63 dims = 21 points × 3 coordinates (meters)**; zeros when not solved (placeholder column) |
|
| 855 |
+
| `observation.gripper_left_force` / `_right_force` | **UMI gripper contact force** `[fx, fy, fz]` in mN (3 dims); the twin column `…_force_ns` carries that sample's capture time (ns, monotonic) |
|
| 856 |
+
| `observation.gripper_left_force_matrix` / `_right_force_matrix` | **Force matrix**, 10×10×3 = 300 values, integer-encoded — the multiplier is per-recording, read `features["observation.…_force_matrix"].scale` in `meta/info.json` (default ×100); twin column `…_force_matrix_ns` = capture time (ns) |
|
| 857 |
+
| `observation.gripper_state` | Gripper state `[open %, gripped flag, fz mN]` (3 dims) |
|
| 858 |
+
| `observation.slam_trajectory` | **Gripper SLAM trajectory** (variable length): 8 values per point `[t, x, y, z, qx, qy, qz, qw]` — position in meters in the post-origin frame, quaternion, `t` monotonic in seconds |
|
| 859 |
+
|
| 860 |
+
> The force/matrix `_ns` columns exist because forces are sampled latest-wins while RGB takes the oldest queued frame, so row numbers alone drift apart within a take; resample by those timestamps to align modalities.
|
| 861 |
+
> With a **second gripper**, every one of its columns gets the `gripper_2_` prefix (e.g. `observation.gripper_2_gripper_left_force`), so two grippers never collide.
|
| 862 |
|
| 863 |
**③ Action & status**
|
| 864 |
|
|
|
|
| 936 |
| 6-2 | 6.1 | One UVC tile (info bar: name/FPS/light) | `docs/images/fig-6-2.png` | Main | 800×450 |
|
| 937 |
| 6-3 | 6.2 | D435 two tiles (RGB + depth heatmap) | `docs/images/fig-6-3.png` | Main | 1200×450 |
|
| 938 |
| 6-4 | 6.3 | Stereo three tiles (left/right/depth) | `docs/images/fig-6-4.png` | Main | 1400×450 |
|
| 939 |
+
| 6-6 | 6.0 | Lite device area close-up (five dropdowns + buttons; the UMI Gripper group appears on Linux packages only) | `docs/images/fig-6-6.png` | Lite | 400×300 |
|
| 940 |
| 7-1 | 7.1/7.2 | Recording in progress (⏱ timer, "Recording: 1", locked panel) | `docs/images/fig-7-1.png` | Main | 1600×900 |
|
| 941 |
| 7-2 | 7.1 | Progress-full refusal dialog | `docs/images/fig-7-2.png` | Main | native |
|
| 942 |
| 7-3 | 7.4 | Leave-confirmation dialog while recording | `docs/images/fig-7-3.png` | Main | native |
|
|
|
|
| 966 |
| Code | Symptom | Solution |
|
| 967 |
|---|---|---|
|
| 968 |
| **A** | Python not found / auto-install failed | ① Offline: put `python-3.12.10-amd64.exe` into `wheels/` and retry; ② Install Python 3.12.x 64-bit manually, tick **Add python.exe to PATH**; ③ A Microsoft Store Python may interfere — uninstall it, install the official one; ④ Python < 3.10 counts as missing |
|
| 969 |
+
| **B** | *(Lite edition on Linux only)* "夹爪原生资源缺失" — the gripper native payload is missing | The Linux Lite package ships the UMI gripper payload (`core/gripper/native/` ~460 MB + `core/gripper/sightac_sdk/`) as a **required** resource; the launcher checks 7 items up front and refuses to start (so you don't sit through a wheel install first). The message lists exactly which items are missing. ① Re-extract the full package (most common cause: one directory level short, or an upgrade that copied code but not the payload); ② Only need D435/UVC/gloves → `KSQ_GRIPPER_SKIP_CHECK=1 ./start_lite.sh` (the gripper group then stays empty); ③ Custom install location → point `KSQ_GRIPPER_NATIVE_ROOT` at the payload |
|
| 970 |
| **C** | venv creation failed | ① Free disk space (needs ~2 GB); ② Antivirus blocked it → whitelist the folder, then `start.bat reinstall`; ③ Path too long → move to a short path like `C:\collector`; ④ Special characters in path → use a plain-English folder |
|
| 971 |
| **C2** | Old venv deletion failed | The main program is still running — close its window, then `start.bat reinstall` |
|
| 972 |
| **D** | Dependency download/install failed | ① Check the network and rerun (partial downloads are cached); ② Corporate firewall → ask admin to allow PyPI mirrors or use the offline route (2.6); ③ Antivirus blocking pip → whitelist; ④ Repeated failures → `start.bat reinstall` |
|
|
|
|
| 1028 |
| Quick-start guide | Yes (pops on first launch) | No |
|
| 1029 |
| Hand skeleton solving | Yes (MANO 21 keypoints filled live) | No (columns stay zero — valid) |
|
| 1030 |
| Theme | qt-material dark theme | Lightweight dark style |
|
| 1031 |
+
| Devices | UVC ≤ 8 + D435 + stereo S80C/S80M + gloves + UMI gripper (Linux) | One of each: D435 / UVC / USB-C glove; BLE gloves 2 (left + right at once); UMI grippers up to 2 (Linux package only) |
|
| 1032 |
+
| Gripper (UMI / Fays S80M) | Full visualization (stereo tiles, force-matrix heatmaps, pose view, open/close panel) | **Records without any visualization** — RGB video + force/tactile/matrix + SLAM trajectory are written to disk, nothing is shown; no "re-read factory calibration" context menu (use `tools/import_gripper_calibration.py --generate` instead) |
|
| 1033 |
+
| Gripper payload (~460 MB) | Ships in the repo (`core/gripper/native/`, not in git) | Ships in the **Linux** package (required; `[Error B]` if missing) — the **Windows** package carries the gripper code but no payload, so its gripper group stays empty there |
|
| 1034 |
| UVC recording | Linux 1280×960, Windows 1280×720 @30 | Fixed 640×480 @30 |
|
| 1035 |
| Preview | Full res, capped 30 fps | 640×360 @15 fps |
|
| 1036 |
| Encoder | Auto probe (nvenc → x265 → x264) | Fixed x264 (low-end friendly) |
|
|
|
|
| 1092 |
| 使用说明.md / 使用说明_EN.md | Quick guide + error-code reference (opened by `start.bat help`) |
|
| 1093 |
| README.md | Repo overview & dev info (hardware table, changelog) |
|
| 1094 |
| docs/ (ui.md, data.md, file_format.md …) | Implementation details, advanced reading |
|
| 1095 |
+
| 使用说明_lite.md | Lite edition quick guide & acceptance checklist |
|
| 1096 |
| tools/glove_package/使用说明.md | Glove toolchain details |
|
| 1097 |
|
| 1098 |
**When reporting a problem**, include these three things to speed things up:
|
|
|
|
| 1197 |
| `run.bat` / `run.sh` | 直接启动(部署完成后日常用它,更快) |
|
| 1198 |
| `main.py` | 主程序入口(不需要手动运行) |
|
| 1199 |
| `data/` | **你录的全部数据 + 本地配置都在这里**,备份就拷贝整个目录 |
|
| 1200 |
+
| `start_lite.bat` / `start_lite.sh` | **极简版**一键部署 + 启动(独立环境 `venv_lite/`),见 2.7 |
|
| 1201 |
+
| `lite_package/` | 极简版发行目录(同一套东西的打包副本,可直接发给只装极简版的机器;Linux 版另含约 460MB 夹爪原生载荷,见 2.7) |
|
| 1202 |
| `wheels/` | 离线安装包(管理员准备的,普通用户没有就忽略) |
|
| 1203 |
| `docs/` | 文档集(开发文档,进阶阅读用) |
|
| 1204 |
|
|
|
|
| 1319 |
|
| 1320 |
| 步骤 | 操作 |
|
| 1321 |
|---|---|
|
| 1322 |
+
| 运行位置 | 含 `main_lite.py` 的目录 —— 本仓库里就是**根目录**(见 1.3);`lite_package/` 是同一套东西的打包副本 |
|
| 1323 |
| 双击 | `start_lite.bat`(Windows)/ `./start_lite.sh`(Linux) |
|
| 1324 |
| 自动完成 | 同样六步:定位 Python → 建独立虚拟环境 `venv_lite/` → 装精简依赖(9 个包)→ 自检 → 启动 |
|
| 1325 |
+
| 体积 | 环境约 750MB,比主程序小;**Linux** 包另含夹爪原生载荷约 460MB(合计约 1.22GB),Windows 包不含 |
|
| 1326 |
+
| 夹爪 | **UMI 夹爪仅 Linux 包支持,且只录不显**(无双目画面、无力矩阵热力图、无位姿轨迹,数据照常落盘)。Windows 包带夹爪代码但不带载荷,「UMI 夹爪」组**框还在但永远是空的**(下拉框无条目,点「开启」提示「列表中没有设备」)——属**预期**,不是故障 |
|
| 1327 |
|
| 1328 |

|
| 1329 |
|
| 1330 |
- 子命令同样支持:`reinstall`(重装)、`help`(打开极简版说明)。
|
| 1331 |
+
- **Linux** 启动时多做一步夹爪资源自检(7 项,`[B]`),缺资源直接报 `[错误 B]` 拒启;只想要 D435/UVC/手套时 `KSQ_GRIPPER_SKIP_CHECK=1 ./start_lite.sh` 跳过(跳过后夹爪组为空,其它设备照常)。
|
| 1332 |
- 离线环境用 `python scripts/pack_wheels.py --lite` 生成 `wheels/` 后同法拷贝。
|
| 1333 |
- **数据互通**:极简版与主程序共用 `data/` 目录——极简版录��数据,主程序照样能回放、上传(见附录 C)。
|
| 1334 |
|
|
|
|
| 1444 |
极简版定位「**连设备 → 采集 → 上传**」三件事,砍掉了登录、任务页、回放等一切多余功能。窗口 980×640,标题「DAQ 极简采集 (lite v1.2.0)」,自上而下:
|
| 1445 |
|
| 1446 |
1. **顶部服务器行**:服务器地址输入框 + 「测试连接」按钮 + 「自动上传」「上传后删除」两个勾选框;
|
| 1447 |
+
2. **左侧设备区**:五组设备(D435 深度相机 / UVC 摄像头 / USB-C 手套 / BLE 手套 / UMI 夹爪——最后一组仅 Linux 包有),每组一个设备下拉框 + 一个「开启」按钮(开启后变成「关闭」;BLE 手套组有两行,可同时连左右两只手套),底部「重新扫描」按钮;
|
| 1448 |
3. **右侧预览区**:640×360 实时画面(15fps),「预览源」下拉可切换 RGB / 深度热力图 / UVC RGB;
|
| 1449 |
4. **录制行**:任务名输入框(可选填)+ 「▶ 开始」(绿)「■ 停止」(红)「⛔ 丢弃」(橙)三个按钮 + 状态文字(待机 / ⏺ 录制中 时长);
|
| 1450 |
5. **上传行**:待上传列表(显示 任务名 —— 状态 (ep n),状态有 未上传/上传中/已上传/失败)+「上传选中」「上传全部未上传」按钮 + 上传进度条;
|
|
|
|
| 1571 |
| 双目相机 S80C / S80M | USB 3.0 | 1 | 采集 50fps → 录制抽帧 30fps | 免装 SDK(程序自带);与 RealSense 不能同开(见 6.3) |
|
| 1572 |
| USB Type-C 手套 | USB 串口(STM32 CDC) | 左右各 1 | 16 传感器 IMU + 16×16 触觉,额定 60fps | 免驱动;串口波特率 115200 |
|
| 1573 |
| BLE 蓝牙手套 | 蓝牙 | 左右各 1 | 16×16 触觉 | 电脑需开蓝牙;广播名含 L/R 自动分左右手 |
|
| 1574 |
+
| UMI 夹爪(Fays S80M) | USB 3.0 + 串口 | 2 台 | 左目 RGB + 10×10×3 力矩阵 + SLAM 轨迹,30fps | **仅 Linux**(原生栈是 ELF);需 460MB 原生载荷(见 2.7 与附录 C) |
|
| 1575 |
|
| 1576 |

|
| 1577 |
|
|
|
|
| 1801 |
|
| 1802 |
极简版的录制只有三个按钮,流程更直白:
|
| 1803 |
|
| 1804 |
+
1. 先在左侧设备区开启至少一台设备——**必须先开 D435、UVC 或 UMI 夹爪(Linux)**:只开手套是录不了的(录制必须有视频源,程序会提示「请先开启 D435 深度相机、UVC 摄像头或 UMI 夹爪」);只开夹爪时以夹爪**左目 RGB** 为主视频源;
|
| 1805 |
2. 任务名输入框选填(不填也能录,数据归到默认任务下);
|
| 1806 |
3. 点 **「▶ 开始」** → 录制中显示「⏺ 录制中 时长」;点 **「■ 停止」** = 保存;点 **「⛔ 丢弃」** = 作废本段;
|
| 1807 |
4. 极简版的固定参数:UVC 录制 640×480@30、预览 640×360@15fps;**编码固定 H.264/x264**(跳过编码器探针,低端机友好);
|
|
|
|
| 1954 |
| `d435_rgb` / `D435_head_depth` | RealSense D435 的彩色 / 深度 |
|
| 1955 |
| `stereo_left` / `stereo_right` | 双目相机左 / 右目 |
|
| 1956 |
| `stereo_depth` | 双目视差深度 |
|
| 1957 |
+
| `gripper_rgb`(第二台夹爪是 `gripper_2_rgb`) | UMI 夹爪左目(开着夹爪就会录;极简版里这是唯一的夹爪视频) |
|
| 1958 |
|
| 1959 |
<a id="cn-ch9-3"></a>
|
| 1960 |
|
|
|
|
| 1984 |
| `observation.<sn>_imu_quat` | USB 手套 IMU:16 传感器 × 4 分量四元数 = 64 维 |
|
| 1985 |
| `observation.<sn>_imu_valid` | 对应 16 个传感器有效性标志(16 维) |
|
| 1986 |
| `observation.left_hand_pose` / `observation.right_hand_pose` | 手部关键点:**63 维 = 21 点 × 3 坐标(米)**;未解算时是 0 占位列 |
|
| 1987 |
+
| `observation.gripper_left_force` / `_right_force` | **UMI 夹爪接触力** `[fx, fy, fz]`(mN,3 维);孪生列 `…_force_ns` 是该样本的采集时刻(纳秒,单调钟) |
|
| 1988 |
+
| `observation.gripper_left_force_matrix` / `_right_force_matrix` | **力矩阵** 10×10×3 = 300 维,整型编码 —— 倍率逐段记录在 `meta/info.json` 的 `features["observation.…_force_matrix"].scale`(默认 ×100);孪生列 `…_force_matrix_ns` = 采集时刻(纳秒) |
|
| 1989 |
+
| `observation.gripper_state` | 夹爪状态 `[开合百分比, 抓握判定, fz(mN)]`(3 维) |
|
| 1990 |
+
| `observation.slam_trajectory` | **夹爪 SLAM 轨迹**(变长):每点 8 值 `[t, x, y, z, qx, qy, qz, qw]` —— 位置是原点相对化后的米制坐标,四元数为姿态,`t` 单调递增(秒) |
|
| 1991 |
+
|
| 1992 |
+
> 力/力矩阵的 `_ns` 孪生列存在的理由:力走 latest-wins 单槽(永远取最新),RGB 走队头最旧帧,两条支路的行号在段内会持续错开;按这些时间戳重采样即可把跨模态对齐压到半帧以内。
|
| 1993 |
+
> 开**第二台夹爪**时,它的所有列统一加 `gripper_2_` 前缀(如 `observation.gripper_2_gripper_left_force`),两台互不串列。
|
| 1994 |
|
| 1995 |
**③ 动作与状态**
|
| 1996 |
|
|
|
|
| 2068 |
| 6-2 | 6.1 UVC | 一路 UVC 画面格(信息条:设备名/FPS/状态灯) | `docs/images/fig-6-2.png` | 普通版 | 800×450 |
|
| 2069 |
| 6-3 | 6.2 RealSense | D435 两格画面(RGB + 深度热力图) | `docs/images/fig-6-3.png` | 普通版 | 1200×450 |
|
| 2070 |
| 6-4 | 6.3 双目 | S80C 三格画面(左目/右目/深度热力图) | `docs/images/fig-6-4.png` | 普通版 | 1400×450 |
|
| 2071 |
+
| 6-6 | 6.0 极简版设备区 | 极简版左列设备区特写(五组下拉框与开启按钮;UMI 夹爪那组仅 Linux 包有) | `docs/images/fig-6-6.png` | 极简版 | 400×300 |
|
| 2072 |
| 7-1 | 7.1/7.2 录制中 | 录制中整窗(⏱计时、录制中: 1、面板锁定态) | `docs/images/fig-7-1.png` | 普通版 | 1600×900 |
|
| 2073 |
| 7-2 | 7.1 拒绝录制 | 「任务进度已满」拒绝弹窗 | `docs/images/fig-7-2.png` | 普通版 | 对话框原生 |
|
| 2074 |
| 7-3 | 7.4 中断确认 | 录制中返回任务选择的确认弹窗 | `docs/images/fig-7-3.png` | 普通版 | 对话框原生 |
|
|
|
|
| 2098 |
| 错误码 | 现象 | 解决方案 |
|
| 2099 |
|---|---|---|
|
| 2100 |
| **A** | 找不到 Python / 自动安装失败 | ① 离线环境:把 `python-3.12.10-amd64.exe` 放入 `wheels/` 后重试;② 手动安装:官网下载 Python 3.12.x **64 位**,安装时勾选 **Add python.exe to PATH**;③ 已装过 Python:可能是 Microsoft Store 版干扰,在「设置-应用」卸载后装官网版;④ 已有 Python 但版本低于 3.10 同样走 ② |
|
| 2101 |
+
| **B** | **夹爪原生资源缺失**(仅极简版 Linux 包;输出里逐项列了缺什么) | 极简版 Linux 包把 UMI 夹爪的原生载荷(`core/gripper/native/` 约 460MB + `core/gripper/sightac_sdk/`)当**必需资源**,启动时先查 7 项,缺了直接拒启(不会白装一遍依赖)。① 最省事:重新完整解压分发包(最常见原因是解压少了一层目录,或覆盖升级时只拷了代码没拷载荷);② 只想要 D435/UVC/手套:`KSQ_GRIPPER_SKIP_CHECK=1 ./start_lite.sh` 跳过(跳过后夹爪组为空,其它设备照常);③ 自定义安装位置:用 `KSQ_GRIPPER_NATIVE_ROOT=<native 目录>` 指路 |
|
| 2102 |
| **C** | 虚拟环境创建失败 | ① 磁盘空间不足(需约 2GB);② 杀毒软件拦截 → 目录加白名单后 `start.bat reinstall`;③ 路径过长 → 移到 `C:\collector` 之类短路径;④ 路径含特殊字符 → 换纯英文目录 |
|
| 2103 |
| **C2** | 旧 venv 删除失败 | 主程序还在运行,先关闭窗口再 `start.bat reinstall` |
|
| 2104 |
| **D** | 依赖下载/安装失败 | ① 网络不通 → 检查网络后重跑(已下载部分有缓存,不重复);② 公司网络限制 → 请管理员放行 PyPI 镜像,或走离线交付(见第 2.6 节);③ 杀毒/防火墙拦 pip → 白名单;④ 多次失败 → `start.bat reinstall` |
|
|
|
|
| 2163 |
| 使用说明向导 | 有(首次启动弹出) | 无 |
|
| 2164 |
| 手部骨架解算 | 有(MANO 21 关键点实时回填) | 无(数据列留 0 占位,合法) |
|
| 2165 |
| 界面主题 | qt-material 暗色主题 | 轻量暗色样式 |
|
| 2166 |
+
| 设备数量 | UVC ≤ 8 路 + D435 + 双目 S80C/S80M + 手套 + UMI 夹爪(Linux) | 每类 1 台:D435 / UVC / USB-C 手套;BLE 手套 2 台(左右手同时);UMI 夹爪最多 2 台(仅 Linux 包) |
|
| 2167 |
+
| 夹爪(UMI / Fays S80M) | 完整可视化(双目画面、力矩阵热力图、位姿轨迹、开合面板) | **只录不显**——RGB 视频 + 力/触觉/力矩阵 + SLAM 轨迹照常落盘,界面上什么都不显示;也没有「重新读取出厂标定」右键菜单(改走 `tools/import_gripper_calibration.py --generate`) |
|
| 2168 |
+
| 夹爪原生载荷(约 460MB) | 仓库自带(`core/gripper/native/`,不入库) | 随 **Linux** 包下发(必需资源,缺了报 `[错误 B]`);**Windows** 包带夹爪代码但不带载荷,那边的夹爪组框在、里面永远为空 |
|
| 2169 |
| UVC 录制分辨率 | Linux 1280×960、Windows 1280×720 @30 | 固定 640×480 @30 |
|
| 2170 |
| 预览 | 全分辨率,30fps 上限 | 640×360 @15fps |
|
| 2171 |
| 编码器 | 自动探测(nvenc → x265 → x264) | 固定 x264(低端机友好) |
|
|
|
|
| 2227 |
| 使用说明.md / 使用说明_EN.md | 快速指引 + 错误码速查(`start.bat help` 打开它) |
|
| 2228 |
| README.md | 仓库总览与开发信息(硬件支持表、更新记录) |
|
| 2229 |
| docs/(ui.md、data.md、file_format.md 等) | 模块与格式实现细节,进阶阅读 |
|
| 2230 |
+
| 使用说明_lite.md | 极简版快速指引与验收清单 |
|
| 2231 |
| tools/glove_package/使用说明.md | 手套识别工具链详细说明 |
|
| 2232 |
|
| 2233 |
**遇到问题反馈时**,请附上三样东西,能大幅加快排查:
|
collector/{lite_package/使用说明_lite.md → 使用说明_lite.md}
RENAMED
|
@@ -3,6 +3,13 @@
|
|
| 3 |
只做三件事:**连接设备 → 采集数据 → 上传**。无登录、无回放、无任务页、无骨架解算。
|
| 4 |
体积小(无 torch/scipy/mediapipe/pyvista)、CPU 要求低(录制直接 x264、预览 15fps)。
|
| 5 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 6 |
> 与主程序共用 `data/`(录制目录、`pipeline.db`、`server_config.json`、手套序列号绑定),
|
| 7 |
> 两版可交替使用、数据互通;但虚拟环境独立(`venv_lite/`,与主程序 `venv/` 互不影响)。
|
| 8 |
|
|
@@ -24,6 +31,7 @@
|
|
| 24 |
| `start_lite.bat help` | 打开本文档 |
|
| 25 |
|
| 26 |
**错误码**:`[错误 A]` 无 Python 且自动安装失败 → 手动装 3.12 并勾选 "Add python.exe to PATH";
|
|
|
|
| 27 |
`[错误 C]` 建 venv 失败 → 磁盘空间/杀毒软件/中文路径;`[错误 D]` 依赖安装失败 → 网络或让管理员
|
| 28 |
生成离线包;`[错误 E]` 自检失败 → 杀毒软件或 `reinstall`;`[错误 F]` 启动即退 → 显卡驱动/远程桌面;
|
| 29 |
`[错误 G]` 解压层次不对 → 保持文件夹完整、不要只拷 bat。
|
|
@@ -32,16 +40,20 @@
|
|
| 32 |
`wheels\` 目录(约几百 MB),连同 `python-3.12.10-amd64.exe` 一起拷到客户项目根目录,
|
| 33 |
`start_lite.bat` 自动改走纯离线安装。
|
| 34 |
|
| 35 |
-
**Linux**:`./start_lite.sh`(`reinstall` / `help` 子命令同上)。
|
|
|
|
| 36 |
|
| 37 |
---
|
| 38 |
|
| 39 |
## 二、日常操作流程
|
| 40 |
|
| 41 |
-
1. **插设备**:D435(USB3)、UVC 摄像头、USB-C 手套、BLE 手套(已配对)
|
|
|
|
|
|
|
| 42 |
2. **开启**:左侧设备组选中设备 → 点「开启」;中上方出现预览(RGB / 深度热力图 / UVC 可切换,
|
| 43 |
开启设备后预览源会自动切到新开的设备)。
|
| 44 |
-
- 录制必须有视频源:**先开 D435
|
|
|
|
| 45 |
- 手套左右手按序列号/广播名 L/R 自动绑定传感器列,与主程序共用绑定记忆。
|
| 46 |
3. **录制**:填任务名(可选)→「▶ 开始」→「■ 停止」保存(「X 丢弃」作废)。
|
| 47 |
保存后按「自动上传」开关决定是否自动入队上传。
|
|
@@ -52,33 +64,101 @@
|
|
| 52 |
|
| 53 |
---
|
| 54 |
|
| 55 |
-
## 三、
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 56 |
|
| 57 |
- 无登录/任务页/回放/查看器/使用向导;无骨架关键点解算(parquet 的 hand_pose 列为全零占位,合法)。
|
| 58 |
-
- 无
|
|
|
|
|
|
|
| 59 |
- UVC 摄像头采集固定 640×480、30fps(预览刷新仍限 15fps;无深度图);
|
| 60 |
Windows 枚举无需额外依赖(pygrabber+comtypes 可选,装了仅提升枚举质量)。
|
| 61 |
- 预览固定 640×360、15fps;编码器固定 x264(跳过 x265 速度探针,低端机友好)。
|
| 62 |
- 依赖只有 9 个包(`requirements-lite.txt`);`imageio-ffmpeg`(77MB)提供内置 ffmpeg,
|
| 63 |
装有系统 ffmpeg 的机器可删除该行再重装以省体积。
|
| 64 |
|
| 65 |
-
##
|
| 66 |
|
| 67 |
- [ ] D435 出现在列表 → 开启 → RGB/深度预览切换正常 → 录 30s → 停止 → `data/recordings/<任务>/`
|
| 68 |
出现 `videos/chunk-000/d435_rgb/`、`depth/`、`data/chunk-000/*.parquet`、`meta/info.json`
|
| 69 |
- [ ] UVC 摄像头出现在列表 → 开启(不开 D435)→ 预览切到 UVC 有画面 → 录 10s → 停止 →
|
| 70 |
`videos/chunk-000/uvc_rgb/` 有视频,parquet 无深度相关列
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 71 |
- [ ] 主程序 ☁ 上传对话框能看到刚才的会话(共用 `pipeline.db`)
|
| 72 |
- [ ] USB 手套开启 → parquet 含 `observation.<列>_imu_quat/_imu_valid`,hand_pose 列全零
|
| 73 |
- [ ] BLE 手套广播名 L/R 绑定左右列正确
|
| 74 |
- [ ] 服务器地址保存后重启仍在(与主程序共用 `server_config.json`,不互相覆盖)
|
| 75 |
- [ ] 勾「自动上传」→ 录制完成后自动上传 → 服务器可见 session
|
| 76 |
-
- [ ] 无 Python 的 Windows 真机跑 `start_lite.bat` 全流程(含 `wheels\` 离线安装)
|
|
|
|
| 77 |
|
| 78 |
-
##
|
| 79 |
|
| 80 |
- **预览显示「无信号」**:开启设备后预览会自动切到新开设备的画面;若仍无信号,
|
| 81 |
看窗口下方日志 —— 出现「无法打开…可能被其他程序占用」说明摄像头正被别的程序
|
| 82 |
使用(主程序 / 浏览器 / 另一个极简版窗口),关掉它们后点「关闭」再重新「开启」。
|
| 83 |
-
- **录不了**:录制必须有视频源 —— 先开 D435
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 84 |
- **改了代码不生效**:`start_lite.sh reinstall` 重装依赖,或先关闭所有旧窗口再重开。
|
|
|
|
| 3 |
只做三件事:**连接设备 → 采集数据 → 上传**。无登录、无回放、无任务页、无骨架解算。
|
| 4 |
体积小(无 torch/scipy/mediapipe/pyvista)、CPU 要求低(录制直接 x264、预览 15fps)。
|
| 5 |
|
| 6 |
+
设备:**D435 / UVC 摄像头 / USB-C 手套 / BLE 手套 / UMI 夹爪**。夹爪**只录不显**
|
| 7 |
+
(数据照常落盘,不做任何夹爪可视化),且**仅 Linux 包**支持 —— Windows 包里「UMI 夹爪」
|
| 8 |
+
组是空的(和没插 D435 时的 D435 组一样),见 [§三](#三夹爪umi--fays-s80m仅-linux-包)。
|
| 9 |
+
|
| 10 |
+
包体积(含首次自动安装的 `venv_lite`,约 750MB):Linux 包约 **1.22GB**(含夹爪原生载荷
|
| 11 |
+
460MB),Windows 包约 **0.75GB**(不含载荷)。
|
| 12 |
+
|
| 13 |
> 与主程序共用 `data/`(录制目录、`pipeline.db`、`server_config.json`、手套序列号绑定),
|
| 14 |
> 两版可交替使用、数据互通;但虚拟环境独立(`venv_lite/`,与主程序 `venv/` 互不影响)。
|
| 15 |
|
|
|
|
| 31 |
| `start_lite.bat help` | 打开本文档 |
|
| 32 |
|
| 33 |
**错误码**:`[错误 A]` 无 Python 且自动安装失败 → 手动装 3.12 并勾选 "Add python.exe to PATH";
|
| 34 |
+
`[错误 B]`(**仅 Linux 包**)夹爪原生资源缺失 → 见 §三,多半是解压少了一层目录;
|
| 35 |
`[错误 C]` 建 venv 失败 → 磁盘空间/杀毒软件/中文路径;`[错误 D]` 依赖安装失败 → 网络或让管理员
|
| 36 |
生成离线包;`[错误 E]` 自检失败 → 杀毒软件或 `reinstall`;`[错误 F]` 启动即退 → 显卡驱动/远程桌面;
|
| 37 |
`[错误 G]` 解压层次不对 → 保持文件夹完整、不要只拷 bat。
|
|
|
|
| 40 |
`wheels\` 目录(约几百 MB),连同 `python-3.12.10-amd64.exe` 一起拷到客户项目根目录,
|
| 41 |
`start_lite.bat` 自动改走纯离线安装。
|
| 42 |
|
| 43 |
+
**Linux**:`./start_lite.sh`(`reinstall` / `help` 子命令同上)。Linux 包**带夹爪支持**,
|
| 44 |
+
启动时会多做一步夹爪原生资源自检(`[B]`),见 §三。
|
| 45 |
|
| 46 |
---
|
| 47 |
|
| 48 |
## 二、日常操作流程
|
| 49 |
|
| 50 |
+
1. **插设备**:D435(USB3)、UVC 摄像头、USB-C 手套、BLE 手套(已配对)、UMI 夹爪
|
| 51 |
+
(USB3 双目 + 串口,仅 Linux 包)。约 2 秒后出现在左侧对应列表;夹爪扫描略慢
|
| 52 |
+
(要经 ESP32 读序列号配对),首次「开启」还要核对/生成出厂标定。
|
| 53 |
2. **开启**:左侧设备组选中设备 → 点「开启」;中上方出现预览(RGB / 深度热力图 / UVC 可切换,
|
| 54 |
开启设备后预览源会自动切到新开的设备)。
|
| 55 |
+
- 录制必须有视频源:**先开 D435 / UVC 摄像头 / 夹爪**(只开 UVC 时以其为主视频源、无深度图;
|
| 56 |
+
只开夹爪时以夹爪**左目 RGB** 为主视频源)。
|
| 57 |
- 手套左右手按序列号/广播名 L/R 自动绑定传感器列,与主程序共用绑定记忆。
|
| 58 |
3. **录制**:填任务名(可选)→「▶ 开始」→「■ 停止」保存(「X 丢弃」作废)。
|
| 59 |
保存后按「自动上传」开关决定是否自动入队上传。
|
|
|
|
| 64 |
|
| 65 |
---
|
| 66 |
|
| 67 |
+
## 三、夹爪(UMI / Fays S80M,仅 Linux 包)
|
| 68 |
+
|
| 69 |
+
夹爪在极简版里是**只录不显**:数据全部照常落盘,但不做任何夹爪可视化(双目画面、
|
| 70 |
+
力矩阵热力图、位姿轨迹、开合力读数都没有 —— 需要看这些请用主程序)。
|
| 71 |
+
|
| 72 |
+
**开启**:插好夹爪 → 左侧「UMI 夹爪」组里选中 → 点「开启」。最多 2 台,按开启顺序
|
| 73 |
+
编号;第二台的列名自动加 `gripper_2_` 前缀(同一台机器上两套数据互不串列)。
|
| 74 |
+
开启后窗口下方日志会打印链路就绪信息;夹爪的**左目 RGB 可当主视频源**,只开夹爪
|
| 75 |
+
也能录(此时 parquet 里没有深度相关列)。
|
| 76 |
+
|
| 77 |
+
**录什么**:
|
| 78 |
+
|
| 79 |
+
| 落盘位置 | 内容 |
|
| 80 |
+
|---|---|
|
| 81 |
+
| `videos/…/gripper_rgb/` | 夹爪左目 RGB 视频 |
|
| 82 |
+
| `observation.gripper_state` | 开合百分比 / 抓握判定 / 法向力 |
|
| 83 |
+
| `observation.gripper_{left,right}_force` | 三轴合力;`_force_ns` 是它的采集时刻 |
|
| 84 |
+
| `observation.gripper_{left,right}_force_matrix` | 10×10×3 力矩阵(整型编码,倍率见 `meta/info.json` 的 `features.*.scale`) |
|
| 85 |
+
| `observation.slam_trajectory` | SLAM 轨迹(8 个值一个点,时间戳单调) |
|
| 86 |
+
|
| 87 |
+
**不存**双目视频与 IMU。录制结束的日志里有一行双目丢帧汇总 —— 只录不显时,那是
|
| 88 |
+
判断链路是否健康的唯一证据,出问题请把它连同 `logs/` 一起发给开发。
|
| 89 |
+
|
| 90 |
+
**为什么需要 460MB**:夹爪的原生栈(`core/gripper/native/` 与 `core/gripper/sightac_sdk/`)
|
| 91 |
+
是**必需资源**,随 Linux 分发包下发,`start_lite.sh` 的 `[B]` 会逐项��验(7 项),
|
| 92 |
+
缺任何一项直接报 `[错误 B]` 并**拒启**(不会白装一遍依赖再失败)。修复办法见报错提示,
|
| 93 |
+
或只想要 D435/UVC/手套时跳过自检:
|
| 94 |
+
|
| 95 |
+
```bash
|
| 96 |
+
KSQ_GRIPPER_SKIP_CHECK=1 ./start_lite.sh # 跳过后夹爪组为空,其它设备照常
|
| 97 |
+
```
|
| 98 |
+
|
| 99 |
+
**换新夹爪 / 重新读取出厂标定**:正常情况下插上新夹爪、点「开启」就会自动读取出厂
|
| 100 |
+
标定并连上,不用手工操作。只有换模组或怀疑标定被写坏时才需要强制重读 —— 极简版
|
| 101 |
+
**没有这个 UI**,用命令行(在部署目录下执行):
|
| 102 |
+
|
| 103 |
+
```bash
|
| 104 |
+
cd <部署目录>
|
| 105 |
+
./venv_lite/bin/python tools/import_gripper_calibration.py --list # 看已覆盖哪些序列号
|
| 106 |
+
./venv_lite/bin/python tools/import_gripper_calibration.py --detect # 现场插着的这只的序列号
|
| 107 |
+
./venv_lite/bin/python tools/import_gripper_calibration.py --generate # 就地为它读一份出厂标定
|
| 108 |
+
```
|
| 109 |
+
|
| 110 |
+
标定是**逐台**的(焦距/基线/IMU 外参各不相同),绝不能跨设备抄;旧文件会先备份成
|
| 111 |
+
`*.bak_<时间戳>`。跑之前先关掉正在用这只夹爪的程序(主程序或极简版窗口)。
|
| 112 |
+
|
| 113 |
+
**Windows**:不支持夹爪(原生栈是 ELF,Windows 上跑不了)。「UMI 夹爪」这一组**还在**
|
| 114 |
+
(五组的框一直都在,和没插 D435 时一样),但里面永远是空的、点「开启」只会提示
|
| 115 |
+
「列表中没有设备」—— 这是**预期**,不是故障。
|
| 116 |
+
|
| 117 |
+
---
|
| 118 |
+
|
| 119 |
+
## 四、与主程序的差异(本版没有的东西)
|
| 120 |
|
| 121 |
- 无登录/任务页/回放/查看器/使用向导;无骨架关键点解算(parquet 的 hand_pose 列为全零占位,合法)。
|
| 122 |
+
- 无独立双目相机 S80C/S80M(那是主程序的设备类型,与夹爪的「只录不显」不是一回事);
|
| 123 |
+
每类设备(D435 / UVC / USB 手套 / BLE 手套 / 夹爪)只支持 1 台(夹爪 2 台)。
|
| 124 |
+
- 无夹爪可视化,也没有「重新读取出厂标定」的右键菜单(改走上面的命令行)。
|
| 125 |
- UVC 摄像头采集固定 640×480、30fps(预览刷新仍限 15fps;无深度图);
|
| 126 |
Windows 枚举无需额外依赖(pygrabber+comtypes 可选,装了仅提升枚举质量)。
|
| 127 |
- 预览固定 640×360、15fps;编码器固定 x264(跳过 x265 速度探针,低端机友好)。
|
| 128 |
- 依赖只有 9 个包(`requirements-lite.txt`);`imageio-ffmpeg`(77MB)提供内置 ffmpeg,
|
| 129 |
装有系统 ffmpeg 的机器可删除该行再重装以省体积。
|
| 130 |
|
| 131 |
+
## 五、手工验收清单(真实设备)
|
| 132 |
|
| 133 |
- [ ] D435 出现在列表 → 开启 → RGB/深度预览切换正常 → 录 30s → 停止 → `data/recordings/<任务>/`
|
| 134 |
出现 `videos/chunk-000/d435_rgb/`、`depth/`、`data/chunk-000/*.parquet`、`meta/info.json`
|
| 135 |
- [ ] UVC 摄像头出现在列表 → 开启(不开 D435)→ 预览切到 UVC 有画面 → 录 10s → 停止 →
|
| 136 |
`videos/chunk-000/uvc_rgb/` 有视频,parquet 无深度相关列
|
| 137 |
+
- [ ] **夹爪**(Linux 包):冷启无 `[错误 B]` → 列表出现「UMI 夹爪 (序列号)」→ 开启 → 日志有
|
| 138 |
+
链路就绪 → **只开夹爪**录 60s → `videos/…/gripper_rgb/` 有视频、`observation.gripper_state`
|
| 139 |
+
与 `gripper_{left,right}_force*`、`slam_trajectory` 列有值、抓握时力非零、轨迹时间戳单调
|
| 140 |
+
- [ ] **夹爪**:录制中拔掉夹爪 USB → 弹窗提示 + 条目录回收 + **录制不中断**(其它设备继续录)
|
| 141 |
+
- [ ] **夹爪**:关窗后 `ps -ef | grep -E "fayssense|ksq-camera-service|fays_vikit"` 零残留
|
| 142 |
+
- [ ] **夹爪**:插第二台夹爪 → 列名走 `gripper_2_*` 前缀、两条 RGB 各自成档、互不串列
|
| 143 |
- [ ] 主程序 ☁ 上传对话框能看到刚才的会话(共用 `pipeline.db`)
|
| 144 |
- [ ] USB 手套开启 → parquet 含 `observation.<列>_imu_quat/_imu_valid`,hand_pose 列全零
|
| 145 |
- [ ] BLE 手套广播名 L/R 绑定左右列正确
|
| 146 |
- [ ] 服务器地址保存后重启仍在(与主程序共用 `server_config.json`,不互相覆盖)
|
| 147 |
- [ ] 勾「自动上传」→ 录制完成后自动上传 → 服务器可见 session
|
| 148 |
+
- [ ] 无 Python 的 Windows 真机跑 `start_lite.bat` 全流程(含 `wheels\` 离线安装),
|
| 149 |
+
确认「UMI 夹爪」组是空的(组框在、无条目)且无任何报错
|
| 150 |
|
| 151 |
+
## 六、常见问题
|
| 152 |
|
| 153 |
- **预览显示「无信号」**:开启设备后预览会自动切到新开设备的画面;若仍无信号,
|
| 154 |
看窗口下方日志 —— 出现「无法打开…可能被其他程序占用」说明摄像头正被别的程序
|
| 155 |
使用(主程序 / 浏览器 / 另一个极简版窗口),关掉它们后点「关闭」再重新「开启」。
|
| 156 |
+
- **录不了**:录制必须有视频源 —— 先开 D435、UVC 摄像头或夹爪。
|
| 157 |
+
- **「UMI 夹爪」组里是空的**:① Windows 或用了 `KSQ_GRIPPER_SKIP_CHECK=1` —— 预期行为
|
| 158 |
+
(组标题还在、下拉框空、点「开启」提示「列表中没有设备」);② Linux 上先看启动时有没有
|
| 159 |
+
`[错误 B]`;③ 夹爪没插好(双目 / 串口都要通),日志里会有扫描失败的原始原因。
|
| 160 |
+
- **`[错误 B]` 夹爪原生资源缺失**:按提示逐项补 `core/gripper/native/`(约 460MB)——
|
| 161 |
+
最常见的原因是解压时少了一层目录,或覆盖升级时只拷了代码没拷载荷。
|
| 162 |
+
- **夹爪开着但没数据**:日志里那条双目丢帧汇总 + `logs/` 目录是排查入口;先确认不是
|
| 163 |
+
被别处的同类进程占着设备(标定/探测程序与运行中的夹爪互斥)。
|
| 164 |
- **改了代码不生效**:`start_lite.sh reinstall` 重装依赖,或先关闭所有旧窗口再重开。
|