Agora board builds (Model Farm, verified on real boards)
Browse filesThis view is limited to 50 files because it contains too many changes. See raw diff
- .gitattributes +28 -0
- LICENSE_MODEL_FARM.pdf +0 -0
- NOTICE.md +5 -0
- README.md +36 -0
- SHA256SUMS +60 -0
- targets/QCS8550/FP16/calibration/sample_000.json +221 -0
- targets/QCS8550/FP16/calibration/sample_001.json +249 -0
- targets/QCS8550/FP16/calibration/sample_002.json +249 -0
- targets/QCS8550/FP16/calibration/sample_003.json +249 -0
- targets/QCS8550/FP16/code/.gitkeep +0 -0
- targets/QCS8550/FP16/code/README.md +150 -0
- targets/QCS8550/FP16/code/python/bevformer.py +611 -0
- targets/QCS8550/FP16/code/python/camera_grid_visualization.py +309 -0
- targets/QCS8550/FP16/code/python/configs/demo_config.json +34 -0
- targets/QCS8550/FP16/code/python/configs/nms_runtime_contract.json +11 -0
- targets/QCS8550/FP16/code/python/datasets/sample4/README.md +10 -0
- targets/QCS8550/FP16/code/python/datasets/sample4/asset_manifest.json +535 -0
- targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_000/cameras/CAM_BACK.jpg +3 -0
- targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_000/cameras/CAM_BACK_LEFT.jpg +3 -0
- targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_000/cameras/CAM_BACK_RIGHT.jpg +3 -0
- targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_000/cameras/CAM_FRONT.jpg +3 -0
- targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_000/cameras/CAM_FRONT_LEFT.jpg +3 -0
- targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_000/cameras/CAM_FRONT_RIGHT.jpg +3 -0
- targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_000/can_bus.raw +0 -0
- targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_000/lidar2img.raw +0 -0
- targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/cameras/CAM_BACK.jpg +3 -0
- targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/cameras/CAM_BACK_LEFT.jpg +3 -0
- targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/cameras/CAM_BACK_RIGHT.jpg +3 -0
- targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/cameras/CAM_FRONT.jpg +3 -0
- targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/cameras/CAM_FRONT_LEFT.jpg +3 -0
- targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/cameras/CAM_FRONT_RIGHT.jpg +3 -0
- targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/can_bus.raw +0 -0
- targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/lidar2img.raw +0 -0
- targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/rotation_can_bus.raw +0 -0
- targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/shift.raw +1 -0
- targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/cameras/CAM_BACK.jpg +3 -0
- targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/cameras/CAM_BACK_LEFT.jpg +3 -0
- targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/cameras/CAM_BACK_RIGHT.jpg +3 -0
- targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/cameras/CAM_FRONT.jpg +3 -0
- targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/cameras/CAM_FRONT_LEFT.jpg +3 -0
- targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/cameras/CAM_FRONT_RIGHT.jpg +3 -0
- targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/can_bus.raw +0 -0
- targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/lidar2img.raw +0 -0
- targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/rotation_can_bus.raw +0 -0
- targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/shift.raw +1 -0
- targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_003/cameras/CAM_BACK.jpg +3 -0
- targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_003/cameras/CAM_BACK_LEFT.jpg +3 -0
- targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_003/cameras/CAM_BACK_RIGHT.jpg +3 -0
- targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_003/cameras/CAM_FRONT.jpg +3 -0
- targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_003/cameras/CAM_FRONT_LEFT.jpg +3 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,31 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_000/cameras/CAM_BACK.jpg filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_000/cameras/CAM_BACK_LEFT.jpg filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_000/cameras/CAM_BACK_RIGHT.jpg filter=lfs diff=lfs merge=lfs -text
|
| 39 |
+
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_000/cameras/CAM_FRONT.jpg filter=lfs diff=lfs merge=lfs -text
|
| 40 |
+
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_000/cameras/CAM_FRONT_LEFT.jpg filter=lfs diff=lfs merge=lfs -text
|
| 41 |
+
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_000/cameras/CAM_FRONT_RIGHT.jpg filter=lfs diff=lfs merge=lfs -text
|
| 42 |
+
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/cameras/CAM_BACK.jpg filter=lfs diff=lfs merge=lfs -text
|
| 43 |
+
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/cameras/CAM_BACK_LEFT.jpg filter=lfs diff=lfs merge=lfs -text
|
| 44 |
+
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/cameras/CAM_BACK_RIGHT.jpg filter=lfs diff=lfs merge=lfs -text
|
| 45 |
+
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/cameras/CAM_FRONT.jpg filter=lfs diff=lfs merge=lfs -text
|
| 46 |
+
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/cameras/CAM_FRONT_LEFT.jpg filter=lfs diff=lfs merge=lfs -text
|
| 47 |
+
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/cameras/CAM_FRONT_RIGHT.jpg filter=lfs diff=lfs merge=lfs -text
|
| 48 |
+
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/cameras/CAM_BACK.jpg filter=lfs diff=lfs merge=lfs -text
|
| 49 |
+
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/cameras/CAM_BACK_LEFT.jpg filter=lfs diff=lfs merge=lfs -text
|
| 50 |
+
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/cameras/CAM_BACK_RIGHT.jpg filter=lfs diff=lfs merge=lfs -text
|
| 51 |
+
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/cameras/CAM_FRONT.jpg filter=lfs diff=lfs merge=lfs -text
|
| 52 |
+
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/cameras/CAM_FRONT_LEFT.jpg filter=lfs diff=lfs merge=lfs -text
|
| 53 |
+
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/cameras/CAM_FRONT_RIGHT.jpg filter=lfs diff=lfs merge=lfs -text
|
| 54 |
+
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_003/cameras/CAM_BACK.jpg filter=lfs diff=lfs merge=lfs -text
|
| 55 |
+
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_003/cameras/CAM_BACK_LEFT.jpg filter=lfs diff=lfs merge=lfs -text
|
| 56 |
+
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_003/cameras/CAM_BACK_RIGHT.jpg filter=lfs diff=lfs merge=lfs -text
|
| 57 |
+
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_003/cameras/CAM_FRONT.jpg filter=lfs diff=lfs merge=lfs -text
|
| 58 |
+
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_003/cameras/CAM_FRONT_LEFT.jpg filter=lfs diff=lfs merge=lfs -text
|
| 59 |
+
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_003/cameras/CAM_FRONT_RIGHT.jpg filter=lfs diff=lfs merge=lfs -text
|
| 60 |
+
targets/QCS8550/FP16/models/QCS8550/FP16/backbone_context.bin.aidem filter=lfs diff=lfs merge=lfs -text
|
| 61 |
+
targets/QCS8550/FP16/models/QCS8550/FP16/decoder_context.bin.aidem filter=lfs diff=lfs merge=lfs -text
|
| 62 |
+
targets/QCS8550/FP16/models/QCS8550/FP16/scene_start_encoder_context.bin.aidem filter=lfs diff=lfs merge=lfs -text
|
| 63 |
+
targets/QCS8550/FP16/models/QCS8550/FP16/temporal_encoder_context.bin.aidem filter=lfs diff=lfs merge=lfs -text
|
LICENSE_MODEL_FARM.pdf
ADDED
|
Binary file (71.1 kB). View file
|
|
|
NOTICE.md
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Provenance
|
| 2 |
+
|
| 3 |
+
Resources are copied without binary changes from the APLUX Model Farm build `registry/BEVFormer-tiny (R50)` that Agora verified on real boards (decoded sample check, then measured), pinned by sha256 in SHA256SUMS.
|
| 4 |
+
|
| 5 |
+
Agora arranges files under targets/<SoC>/<precision>/ and adds model.json. AidLite, AidGen, AidVoice and the Qualcomm QNN runtime are separate dependencies with their own terms.
|
README.md
ADDED
|
@@ -0,0 +1,36 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: other
|
| 3 |
+
license_name: aplux-model-farm
|
| 4 |
+
license_link: https://huggingface.co/APLUX-Agora/BEVFormer-tiny-R50/blob/main/LICENSE_MODEL_FARM.pdf
|
| 5 |
+
tags: [Agora, QNN, AIoT, Qualcomm]
|
| 6 |
+
---
|
| 7 |
+
|
| 8 |
+
# BEVFormer-tiny (R50) model resources
|
| 9 |
+
|
| 10 |
+
输入六路相机与车辆标定,使用 NPU 进行鸟瞰空间编码和三维目标检测;支持显式上一帧状态。
|
| 11 |
+
|
| 12 |
+
Board builds that [Agora](https://mega.aidlux.com) runs and measures on real Qualcomm boards. Try it online, without a board: https://mega.aidlux.com/models/bevformer-tiny-r50
|
| 13 |
+
|
| 14 |
+
| SoC · board | Precisions | Measured on Agora (default precision) |
|
| 15 |
+
| --- | --- | --- |
|
| 16 |
+
| QCS8550 · 犀牛派 X1 (Rhino Pi X1) | FP16 | 402.1 ms |
|
| 17 |
+
|
| 18 |
+
Task: 其他. Inputs: front (image), front_right (image), front_left (image), back (image), back_left (image), back_right (image), calibration (text), history (json). Outputs: image (image), out (json), history (json).
|
| 19 |
+
|
| 20 |
+
## Download
|
| 21 |
+
|
| 22 |
+
```bash
|
| 23 |
+
export AGORA_WORKSPACE="$HOME/my-bevformer-tiny-r50"
|
| 24 |
+
export AGORA_SOC='QCS8550'
|
| 25 |
+
hf download APLUX-Agora/BEVFormer-tiny-R50 \
|
| 26 |
+
--include "targets/$AGORA_SOC/*" --local-dir "$AGORA_WORKSPACE/resources"
|
| 27 |
+
```
|
| 28 |
+
|
| 29 |
+
Each `targets/<SoC>/` holds `model.json` (default precision and variant paths) and one directory per precision. Context binaries are named `model.bin`; multi-file packages are kept as published by Model Farm.
|
| 30 |
+
|
| 31 |
+
Pre- and post-processing, the runner and an exported project: open the model on Agora and choose “部署指南”, or export any workflow that uses it.
|
| 32 |
+
|
| 33 |
+
## Licence and provenance
|
| 34 |
+
|
| 35 |
+
Source model licence: see the original model's terms.
|
| 36 |
+
Converted, deployable resources: APLUX Model Farm License (LICENSE_MODEL_FARM.pdf). See NOTICE.md and SHA256SUMS.
|
SHA256SUMS
ADDED
|
@@ -0,0 +1,60 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
35def1b2da02f8d1b3f5b5dcbb40fb74ddd5074f6dc16f9dd2398509d0d0e6d0 targets/QCS8550/FP16/calibration/sample_000.json
|
| 2 |
+
d37b43abb24770364bc0f344d1208e5256de9a3f85c5f9b6e3f6cf18db1d094b targets/QCS8550/FP16/calibration/sample_001.json
|
| 3 |
+
89c891d43cc43e3665e32fec157b861d12d8fd38324a4ae7a5a8350290ef7c83 targets/QCS8550/FP16/calibration/sample_002.json
|
| 4 |
+
80052246ff0d259bbcb0b719a0a0c86909808a6f993b94999deef26085ef0920 targets/QCS8550/FP16/calibration/sample_003.json
|
| 5 |
+
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 targets/QCS8550/FP16/code/.gitkeep
|
| 6 |
+
1555a8c1d29dfde3c920830faca6eda8cbc6ba36eaf0888379de6a845336380c targets/QCS8550/FP16/code/README.md
|
| 7 |
+
3b3f71b3b61a91a85b63a6b8c7257437943fcd97fef6b2b453ffcb7a50d0a0ae targets/QCS8550/FP16/code/python/bevformer.py
|
| 8 |
+
bfe8933e69529b417012db2bc5c01a8daad3b86240a269f08eb8fc27f0bfc689 targets/QCS8550/FP16/code/python/camera_grid_visualization.py
|
| 9 |
+
2be061c394e1bf4054d6eed07d166f1d5f97502c91b5c679d47943b378a26494 targets/QCS8550/FP16/code/python/configs/demo_config.json
|
| 10 |
+
492eb30a0051e11b5bbbfd454c0c4eb9fb27e50259a9651866341e90f2b9ffd6 targets/QCS8550/FP16/code/python/configs/nms_runtime_contract.json
|
| 11 |
+
e50d0fc8c46c0ba42578a508d5c9acba16345dc77d172567cd8c0400199c7cab targets/QCS8550/FP16/code/python/datasets/sample4/README.md
|
| 12 |
+
14a9f01da4dc57ac2f57e43e9a68a9ee10c5c38352e721bf2f0665b103f5c9aa targets/QCS8550/FP16/code/python/datasets/sample4/asset_manifest.json
|
| 13 |
+
047dd43f3f9fb7cc426b1d4ae369181242c5555b84baff5d7e1cbd0dc8b07259 targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_000/cameras/CAM_BACK.jpg
|
| 14 |
+
27b2826991733662e4f17c509ad5a66e85e0ca66a6413063d8db9645fa608c11 targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_000/cameras/CAM_BACK_LEFT.jpg
|
| 15 |
+
a998caf0e327816b82d2a142d2581a8c1f9c4e67d50b4af3718a381337b7e415 targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_000/cameras/CAM_BACK_RIGHT.jpg
|
| 16 |
+
0e2067a1618c4424050c45c393ada8c9dbf130fff8d6015004fce901dc07984d targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_000/cameras/CAM_FRONT.jpg
|
| 17 |
+
8c52f73a3ef06a129c27ef51ac65704428caebfaa60c52f14a5953bbb0124a18 targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_000/cameras/CAM_FRONT_LEFT.jpg
|
| 18 |
+
15cf59c79b086d14da6fae5a7ee5b0f4d0fac0c0a22c0a85faaa5fb215521191 targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_000/cameras/CAM_FRONT_RIGHT.jpg
|
| 19 |
+
974730a9cf60a4a0cae487ed4b296faf6621abec0bcd7276020ade042fd8dc3b targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_000/can_bus.raw
|
| 20 |
+
f1df04f672d707f2fbc8ea9799ee84d8795a1fe22f17ed89c08327e63f663501 targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_000/lidar2img.raw
|
| 21 |
+
fdbdaa77e60ed5a6a762a4ea9a8eb1cb2ca9a3d94c610a55a2399a05c04b6ba6 targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/cameras/CAM_BACK.jpg
|
| 22 |
+
000f7943769e91ffa842a7dfd1bfc31e0c3f4b37325a98846e8261304028f2ad targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/cameras/CAM_BACK_LEFT.jpg
|
| 23 |
+
39d28a8a5a6f61370a7e7e1e965a6d8610b571195c50fd29da960e1fb2fbee80 targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/cameras/CAM_BACK_RIGHT.jpg
|
| 24 |
+
d5624fcecf12310c3d0760362e097c38cdcdce987e032aaf2883071cd14e2732 targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/cameras/CAM_FRONT.jpg
|
| 25 |
+
0fe15c0581db41e084df27acb2183b0ab9b0fd36da136ef1b4bcbda6f1a84c58 targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/cameras/CAM_FRONT_LEFT.jpg
|
| 26 |
+
dd64d7463bc06bfd5d5378bdd494b40d68d0894307f3ed971eaaaf131022cf68 targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/cameras/CAM_FRONT_RIGHT.jpg
|
| 27 |
+
8ece2acd67155365b9394e1569b8242d69f3d9805cea19a5153cb51e79e55152 targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/can_bus.raw
|
| 28 |
+
63d7bcbf3a9c63e1184947d9e5112fe30bf4e8533ea29df44daeef3cb610a2ed targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/lidar2img.raw
|
| 29 |
+
741d75aad896829e7246d446635bc21331242c7e28ef869f986d70e048d40962 targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/rotation_can_bus.raw
|
| 30 |
+
530492c17f13ce48ec402622565d7a4ba0987a57b988a3c4d824cc4987dfd5fc targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/shift.raw
|
| 31 |
+
a9a05b52aea16ccf730447170eb836697c26928a112a74249caf423d91d28306 targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/cameras/CAM_BACK.jpg
|
| 32 |
+
a28438e9ff68ccab2ce3a8da4151591a2b15780aecdc21185a911318ed923f72 targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/cameras/CAM_BACK_LEFT.jpg
|
| 33 |
+
1578ba72afd2cc91abb1295c3404a432e41ffc7385b4031a43b586acfb693872 targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/cameras/CAM_BACK_RIGHT.jpg
|
| 34 |
+
0bb82b89f9e6ce47d9e1bcd747b01fe3fd342ccc67eb7a9c693a3fc3fc73a6bc targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/cameras/CAM_FRONT.jpg
|
| 35 |
+
4782aaa9180ef1fe153e702031a3249ada09bdb056ad13ff69abaa2f9e59e35a targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/cameras/CAM_FRONT_LEFT.jpg
|
| 36 |
+
70856b9d713c279b4b3d5911be30944f339760cc81da5a8f2c4d90a20276c1d3 targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/cameras/CAM_FRONT_RIGHT.jpg
|
| 37 |
+
e8badd418a0eb1905986c2a2a5e87fa6e16532dc4899cfff93f47dbe51bed9be targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/can_bus.raw
|
| 38 |
+
7d524fefbeec4c1014ab2773b722aaa13774cb4787fb950e27b1b4de7cc3d403 targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/lidar2img.raw
|
| 39 |
+
a08e149204cd22c460e8b5cc62726de1b29d22fb05b6c33b450e1f94a162d3c1 targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/rotation_can_bus.raw
|
| 40 |
+
9cd32a1a7fbb67a9841ebee6acf497db1fa6142ec9e3ec5c9bb1596dd8be81e8 targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/shift.raw
|
| 41 |
+
34e84e10e04ab229864bd0ddd3067442c0b58e249eeb48088ab417cb259b4985 targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_003/cameras/CAM_BACK.jpg
|
| 42 |
+
cdf1f064c61add58e17d7011713aebbe40be9d7e8290c8816f585fca3841e033 targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_003/cameras/CAM_BACK_LEFT.jpg
|
| 43 |
+
eae354ba8407662caa6c50934b6ee306a4e5aaaeec0b0f2e40c2f6b481485d3f targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_003/cameras/CAM_BACK_RIGHT.jpg
|
| 44 |
+
54384bc8c4896f349bdc643b573f398fbb03ee842ac2af2af2c602a62f48f79d targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_003/cameras/CAM_FRONT.jpg
|
| 45 |
+
1f20956c425bdc84da1bf2610b7214d48930652782394aa0f1f3bef94ce0be35 targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_003/cameras/CAM_FRONT_LEFT.jpg
|
| 46 |
+
4c5209d28a7e3a5f29bd7c02c29bd0f53c669afc43b17af2db098371a9711dc1 targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_003/cameras/CAM_FRONT_RIGHT.jpg
|
| 47 |
+
dbc4945c65d5804131f75f04578e3fbc288330ff5fa977f224eda07eb96e463a targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_003/can_bus.raw
|
| 48 |
+
64cd7874438918c64f54414d1b873b0b0fcf05b2b601f2014c4fe5a8316e7a2c targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_003/lidar2img.raw
|
| 49 |
+
3c88129317e663a14f4ba55a3c0c3cc25e9ec2cd4c497276567e4ecfed8a9ab6 targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_003/rotation_can_bus.raw
|
| 50 |
+
8cc4c98a9615a420d40104ae51d4f1f50b719a9cfec4be8d1b65b4b704517def targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_003/shift.raw
|
| 51 |
+
0c9b733c69c569bc02ede7f3684781ef7a0d7b4e881889f8af87ad449374ba41 targets/QCS8550/FP16/code/python/docs/BOARD_ENVIRONMENT.md
|
| 52 |
+
4f1d35e184f45b0ccdc9ae3484caa5236d2907a6accef0b0d8fee768de2476a4 targets/QCS8550/FP16/code/python/portable_numpy_nmsfreecoder.py
|
| 53 |
+
0026e5f8b3d5d9951f788ea52f374dd55568e18fa4654a647a3474751859dc5f targets/QCS8550/FP16/code/python/run_demo.py
|
| 54 |
+
319094897ee71ede6b7892825d9e6e38f5226361cc998d5e8a6f9fd27d6b760a targets/QCS8550/FP16/code/python/run_test.py
|
| 55 |
+
eb3a441d362e7c1e2eaee9e3c2b9552232d1e8a36af41d14aefbcf1a4af3b9e3 targets/QCS8550/FP16/code/python/utils.py
|
| 56 |
+
ec47d6d7bd68607b805a7b16162eaa012988bef8d845e62e62711be1730e1c28 targets/QCS8550/FP16/composite.json
|
| 57 |
+
9a7a5137a2ed3072cdc2c567822a2316b9501a591ffa204b17568f6c80d8844e targets/QCS8550/FP16/models/QCS8550/FP16/backbone_context.bin.aidem
|
| 58 |
+
f6888090751bf1ff6ecbaef966b3b1a6c97793784f921001b493893a8ce6196c targets/QCS8550/FP16/models/QCS8550/FP16/decoder_context.bin.aidem
|
| 59 |
+
eed541fa4138a177432e2290b4131799d88534005789bef9962e2bbb9e008186 targets/QCS8550/FP16/models/QCS8550/FP16/scene_start_encoder_context.bin.aidem
|
| 60 |
+
25f0063fc7d1b125522d3ac9c53b26a9e270d504fc6c908ae5737ec78ad65b94 targets/QCS8550/FP16/models/QCS8550/FP16/temporal_encoder_context.bin.aidem
|
targets/QCS8550/FP16/calibration/sample_000.json
ADDED
|
@@ -0,0 +1,221 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"cameraOrder": [
|
| 3 |
+
"front",
|
| 4 |
+
"front_right",
|
| 5 |
+
"front_left",
|
| 6 |
+
"back",
|
| 7 |
+
"back_left",
|
| 8 |
+
"back_right"
|
| 9 |
+
],
|
| 10 |
+
"canBus": [
|
| 11 |
+
[
|
| 12 |
+
0.0,
|
| 13 |
+
0.0,
|
| 14 |
+
0.0,
|
| 15 |
+
-0.96875,
|
| 16 |
+
-0.96875,
|
| 17 |
+
-0.96875,
|
| 18 |
+
-0.96875,
|
| 19 |
+
-0.60693359375,
|
| 20 |
+
-0.07635498046875,
|
| 21 |
+
9.875,
|
| 22 |
+
-0.021087646484375,
|
| 23 |
+
-0.0124359130859375,
|
| 24 |
+
-0.0230712890625,
|
| 25 |
+
8.5625,
|
| 26 |
+
0.0,
|
| 27 |
+
0.0,
|
| 28 |
+
5.78125,
|
| 29 |
+
0.0
|
| 30 |
+
]
|
| 31 |
+
],
|
| 32 |
+
"frame": 0,
|
| 33 |
+
"imageSizes": [
|
| 34 |
+
[
|
| 35 |
+
900,
|
| 36 |
+
1600
|
| 37 |
+
],
|
| 38 |
+
[
|
| 39 |
+
900,
|
| 40 |
+
1600
|
| 41 |
+
],
|
| 42 |
+
[
|
| 43 |
+
900,
|
| 44 |
+
1600
|
| 45 |
+
],
|
| 46 |
+
[
|
| 47 |
+
900,
|
| 48 |
+
1600
|
| 49 |
+
],
|
| 50 |
+
[
|
| 51 |
+
900,
|
| 52 |
+
1600
|
| 53 |
+
],
|
| 54 |
+
[
|
| 55 |
+
900,
|
| 56 |
+
1600
|
| 57 |
+
]
|
| 58 |
+
],
|
| 59 |
+
"lidar2img": [
|
| 60 |
+
[
|
| 61 |
+
[
|
| 62 |
+
[
|
| 63 |
+
621.5,
|
| 64 |
+
420.25,
|
| 65 |
+
16.375,
|
| 66 |
+
-177.125
|
| 67 |
+
],
|
| 68 |
+
[
|
| 69 |
+
-9.1015625,
|
| 70 |
+
268.5,
|
| 71 |
+
-613.0,
|
| 72 |
+
-322.25
|
| 73 |
+
],
|
| 74 |
+
[
|
| 75 |
+
-0.0117034912109375,
|
| 76 |
+
0.99853515625,
|
| 77 |
+
0.05401611328125,
|
| 78 |
+
-0.42529296875
|
| 79 |
+
],
|
| 80 |
+
[
|
| 81 |
+
0.0,
|
| 82 |
+
0.0,
|
| 83 |
+
0.0,
|
| 84 |
+
1.0
|
| 85 |
+
]
|
| 86 |
+
],
|
| 87 |
+
[
|
| 88 |
+
[
|
| 89 |
+
682.5,
|
| 90 |
+
-309.75,
|
| 91 |
+
-20.171875,
|
| 92 |
+
-230.875
|
| 93 |
+
],
|
| 94 |
+
[
|
| 95 |
+
189.75,
|
| 96 |
+
160.125,
|
| 97 |
+
-620.0,
|
| 98 |
+
-346.25
|
| 99 |
+
],
|
| 100 |
+
[
|
| 101 |
+
0.84326171875,
|
| 102 |
+
0.5361328125,
|
| 103 |
+
0.03216552734375,
|
| 104 |
+
-0.6103515625
|
| 105 |
+
],
|
| 106 |
+
[
|
| 107 |
+
0.0,
|
| 108 |
+
0.0,
|
| 109 |
+
0.0,
|
| 110 |
+
1.0
|
| 111 |
+
]
|
| 112 |
+
],
|
| 113 |
+
[
|
| 114 |
+
[
|
| 115 |
+
16.1875,
|
| 116 |
+
751.5,
|
| 117 |
+
38.8125,
|
| 118 |
+
-151.25
|
| 119 |
+
],
|
| 120 |
+
[
|
| 121 |
+
-194.625,
|
| 122 |
+
160.25,
|
| 123 |
+
-618.5,
|
| 124 |
+
-339.75
|
| 125 |
+
],
|
| 126 |
+
[
|
| 127 |
+
-0.8232421875,
|
| 128 |
+
0.56591796875,
|
| 129 |
+
0.041229248046875,
|
| 130 |
+
-0.52978515625
|
| 131 |
+
],
|
| 132 |
+
[
|
| 133 |
+
0.0,
|
| 134 |
+
0.0,
|
| 135 |
+
0.0,
|
| 136 |
+
1.0
|
| 137 |
+
]
|
| 138 |
+
],
|
| 139 |
+
[
|
| 140 |
+
[
|
| 141 |
+
-402.0,
|
| 142 |
+
-425.25,
|
| 143 |
+
-13.21875,
|
| 144 |
+
-435.5
|
| 145 |
+
],
|
| 146 |
+
[
|
| 147 |
+
-5.41015625,
|
| 148 |
+
-222.625,
|
| 149 |
+
-407.5,
|
| 150 |
+
-354.25
|
| 151 |
+
],
|
| 152 |
+
[
|
| 153 |
+
-0.008331298828125,
|
| 154 |
+
-0.9990234375,
|
| 155 |
+
-0.039093017578125,
|
| 156 |
+
-1.0166015625
|
| 157 |
+
],
|
| 158 |
+
[
|
| 159 |
+
0.0,
|
| 160 |
+
0.0,
|
| 161 |
+
0.0,
|
| 162 |
+
1.0
|
| 163 |
+
]
|
| 164 |
+
],
|
| 165 |
+
[
|
| 166 |
+
[
|
| 167 |
+
-593.5,
|
| 168 |
+
461.75,
|
| 169 |
+
26.625,
|
| 170 |
+
-312.75
|
| 171 |
+
],
|
| 172 |
+
[
|
| 173 |
+
-231.375,
|
| 174 |
+
-51.28125,
|
| 175 |
+
-626.0,
|
| 176 |
+
-281.0
|
| 177 |
+
],
|
| 178 |
+
[
|
| 179 |
+
-0.94775390625,
|
| 180 |
+
-0.319580078125,
|
| 181 |
+
0.003170013427734375,
|
| 182 |
+
-0.4326171875
|
| 183 |
+
],
|
| 184 |
+
[
|
| 185 |
+
0.0,
|
| 186 |
+
0.0,
|
| 187 |
+
0.0,
|
| 188 |
+
1.0
|
| 189 |
+
]
|
| 190 |
+
],
|
| 191 |
+
[
|
| 192 |
+
[
|
| 193 |
+
142.625,
|
| 194 |
+
-734.5,
|
| 195 |
+
-29.78125,
|
| 196 |
+
-136.25
|
| 197 |
+
],
|
| 198 |
+
[
|
| 199 |
+
222.375,
|
| 200 |
+
-61.40625,
|
| 201 |
+
-625.0,
|
| 202 |
+
-294.0
|
| 203 |
+
],
|
| 204 |
+
[
|
| 205 |
+
0.923828125,
|
| 206 |
+
-0.38232421875,
|
| 207 |
+
-0.0037097930908203125,
|
| 208 |
+
-0.464599609375
|
| 209 |
+
],
|
| 210 |
+
[
|
| 211 |
+
0.0,
|
| 212 |
+
0.0,
|
| 213 |
+
0.0,
|
| 214 |
+
1.0
|
| 215 |
+
]
|
| 216 |
+
]
|
| 217 |
+
]
|
| 218 |
+
],
|
| 219 |
+
"sceneId": "fcbccedd61424f1b85dcbf8f897f9754",
|
| 220 |
+
"version": 1
|
| 221 |
+
}
|
targets/QCS8550/FP16/calibration/sample_001.json
ADDED
|
@@ -0,0 +1,249 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"cameraOrder": [
|
| 3 |
+
"front",
|
| 4 |
+
"front_right",
|
| 5 |
+
"front_left",
|
| 6 |
+
"back",
|
| 7 |
+
"back_left",
|
| 8 |
+
"back_right"
|
| 9 |
+
],
|
| 10 |
+
"canBus": [
|
| 11 |
+
[
|
| 12 |
+
3.705078125,
|
| 13 |
+
-2.103515625,
|
| 14 |
+
0.0,
|
| 15 |
+
-0.96630859375,
|
| 16 |
+
-0.96630859375,
|
| 17 |
+
-0.96630859375,
|
| 18 |
+
-0.96630859375,
|
| 19 |
+
-0.59765625,
|
| 20 |
+
-0.323486328125,
|
| 21 |
+
9.6328125,
|
| 22 |
+
0.01068878173828125,
|
| 23 |
+
-0.01297760009765625,
|
| 24 |
+
-0.0445556640625,
|
| 25 |
+
8.3515625,
|
| 26 |
+
0.0,
|
| 27 |
+
0.0,
|
| 28 |
+
5.76171875,
|
| 29 |
+
-1.03515625
|
| 30 |
+
]
|
| 31 |
+
],
|
| 32 |
+
"frame": 1,
|
| 33 |
+
"imageSizes": [
|
| 34 |
+
[
|
| 35 |
+
900,
|
| 36 |
+
1600
|
| 37 |
+
],
|
| 38 |
+
[
|
| 39 |
+
900,
|
| 40 |
+
1600
|
| 41 |
+
],
|
| 42 |
+
[
|
| 43 |
+
900,
|
| 44 |
+
1600
|
| 45 |
+
],
|
| 46 |
+
[
|
| 47 |
+
900,
|
| 48 |
+
1600
|
| 49 |
+
],
|
| 50 |
+
[
|
| 51 |
+
900,
|
| 52 |
+
1600
|
| 53 |
+
],
|
| 54 |
+
[
|
| 55 |
+
900,
|
| 56 |
+
1600
|
| 57 |
+
]
|
| 58 |
+
],
|
| 59 |
+
"lidar2img": [
|
| 60 |
+
[
|
| 61 |
+
[
|
| 62 |
+
[
|
| 63 |
+
621.5,
|
| 64 |
+
420.5,
|
| 65 |
+
17.203125,
|
| 66 |
+
-180.25
|
| 67 |
+
],
|
| 68 |
+
[
|
| 69 |
+
-8.609375,
|
| 70 |
+
268.75,
|
| 71 |
+
-612.5,
|
| 72 |
+
-324.5
|
| 73 |
+
],
|
| 74 |
+
[
|
| 75 |
+
-0.012176513671875,
|
| 76 |
+
0.99853515625,
|
| 77 |
+
0.05450439453125,
|
| 78 |
+
-0.436279296875
|
| 79 |
+
],
|
| 80 |
+
[
|
| 81 |
+
0.0,
|
| 82 |
+
0.0,
|
| 83 |
+
0.0,
|
| 84 |
+
1.0
|
| 85 |
+
]
|
| 86 |
+
],
|
| 87 |
+
[
|
| 88 |
+
[
|
| 89 |
+
682.5,
|
| 90 |
+
-309.5,
|
| 91 |
+
-19.703125,
|
| 92 |
+
-226.75
|
| 93 |
+
],
|
| 94 |
+
[
|
| 95 |
+
190.25,
|
| 96 |
+
160.5,
|
| 97 |
+
-619.5,
|
| 98 |
+
-346.5
|
| 99 |
+
],
|
| 100 |
+
[
|
| 101 |
+
0.84326171875,
|
| 102 |
+
0.53662109375,
|
| 103 |
+
0.033172607421875,
|
| 104 |
+
-0.61328125
|
| 105 |
+
],
|
| 106 |
+
[
|
| 107 |
+
0.0,
|
| 108 |
+
0.0,
|
| 109 |
+
0.0,
|
| 110 |
+
1.0
|
| 111 |
+
]
|
| 112 |
+
],
|
| 113 |
+
[
|
| 114 |
+
[
|
| 115 |
+
15.640625,
|
| 116 |
+
751.5,
|
| 117 |
+
39.21875,
|
| 118 |
+
-161.0
|
| 119 |
+
],
|
| 120 |
+
[
|
| 121 |
+
-194.125,
|
| 122 |
+
160.375,
|
| 123 |
+
-619.0,
|
| 124 |
+
-341.5
|
| 125 |
+
],
|
| 126 |
+
[
|
| 127 |
+
-0.82373046875,
|
| 128 |
+
0.5654296875,
|
| 129 |
+
0.040618896484375,
|
| 130 |
+
-0.53955078125
|
| 131 |
+
],
|
| 132 |
+
[
|
| 133 |
+
0.0,
|
| 134 |
+
0.0,
|
| 135 |
+
0.0,
|
| 136 |
+
1.0
|
| 137 |
+
]
|
| 138 |
+
],
|
| 139 |
+
[
|
| 140 |
+
[
|
| 141 |
+
-402.0,
|
| 142 |
+
-425.5,
|
| 143 |
+
-13.4296875,
|
| 144 |
+
-435.5
|
| 145 |
+
],
|
| 146 |
+
[
|
| 147 |
+
-5.23046875,
|
| 148 |
+
-222.625,
|
| 149 |
+
-407.5,
|
| 150 |
+
-354.0
|
| 151 |
+
],
|
| 152 |
+
[
|
| 153 |
+
-0.00823211669921875,
|
| 154 |
+
-0.9990234375,
|
| 155 |
+
-0.039215087890625,
|
| 156 |
+
-1.015625
|
| 157 |
+
],
|
| 158 |
+
[
|
| 159 |
+
0.0,
|
| 160 |
+
0.0,
|
| 161 |
+
0.0,
|
| 162 |
+
1.0
|
| 163 |
+
]
|
| 164 |
+
],
|
| 165 |
+
[
|
| 166 |
+
[
|
| 167 |
+
-593.5,
|
| 168 |
+
461.75,
|
| 169 |
+
26.625,
|
| 170 |
+
-310.75
|
| 171 |
+
],
|
| 172 |
+
[
|
| 173 |
+
-231.25,
|
| 174 |
+
-51.21875,
|
| 175 |
+
-626.5,
|
| 176 |
+
-281.25
|
| 177 |
+
],
|
| 178 |
+
[
|
| 179 |
+
-0.94775390625,
|
| 180 |
+
-0.3193359375,
|
| 181 |
+
0.0030879974365234375,
|
| 182 |
+
-0.43359375
|
| 183 |
+
],
|
| 184 |
+
[
|
| 185 |
+
0.0,
|
| 186 |
+
0.0,
|
| 187 |
+
0.0,
|
| 188 |
+
1.0
|
| 189 |
+
]
|
| 190 |
+
],
|
| 191 |
+
[
|
| 192 |
+
[
|
| 193 |
+
142.75,
|
| 194 |
+
-734.5,
|
| 195 |
+
-29.859375,
|
| 196 |
+
-130.75
|
| 197 |
+
],
|
| 198 |
+
[
|
| 199 |
+
222.875,
|
| 200 |
+
-61.21875,
|
| 201 |
+
-625.0,
|
| 202 |
+
-293.0
|
| 203 |
+
],
|
| 204 |
+
[
|
| 205 |
+
0.92431640625,
|
| 206 |
+
-0.382080078125,
|
| 207 |
+
-0.0031280517578125,
|
| 208 |
+
-0.46044921875
|
| 209 |
+
],
|
| 210 |
+
[
|
| 211 |
+
0.0,
|
| 212 |
+
0.0,
|
| 213 |
+
0.0,
|
| 214 |
+
1.0
|
| 215 |
+
]
|
| 216 |
+
]
|
| 217 |
+
]
|
| 218 |
+
],
|
| 219 |
+
"rotationCanBus": [
|
| 220 |
+
[
|
| 221 |
+
3.7056884765625,
|
| 222 |
+
-2.103759765625,
|
| 223 |
+
0.0,
|
| 224 |
+
-0.9663883447647095,
|
| 225 |
+
-0.003500749357044697,
|
| 226 |
+
-0.008033331483602524,
|
| 227 |
+
0.25693726539611816,
|
| 228 |
+
-0.5976930856704712,
|
| 229 |
+
-0.32353678345680237,
|
| 230 |
+
9.631071090698242,
|
| 231 |
+
0.010688905604183674,
|
| 232 |
+
-0.01298072375357151,
|
| 233 |
+
-0.04455821216106415,
|
| 234 |
+
8.349747657775879,
|
| 235 |
+
0.0,
|
| 236 |
+
0.0,
|
| 237 |
+
5.763484477996826,
|
| 238 |
+
-1.035308837890625
|
| 239 |
+
]
|
| 240 |
+
],
|
| 241 |
+
"sceneId": "fcbccedd61424f1b85dcbf8f897f9754",
|
| 242 |
+
"shift": [
|
| 243 |
+
[
|
| 244 |
+
-0.0001398324966430664,
|
| 245 |
+
0.0416259765625
|
| 246 |
+
]
|
| 247 |
+
],
|
| 248 |
+
"version": 1
|
| 249 |
+
}
|
targets/QCS8550/FP16/calibration/sample_002.json
ADDED
|
@@ -0,0 +1,249 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"cameraOrder": [
|
| 3 |
+
"front",
|
| 4 |
+
"front_right",
|
| 5 |
+
"front_left",
|
| 6 |
+
"back",
|
| 7 |
+
"back_left",
|
| 8 |
+
"back_right"
|
| 9 |
+
],
|
| 10 |
+
"canBus": [
|
| 11 |
+
[
|
| 12 |
+
3.62890625,
|
| 13 |
+
-2.171875,
|
| 14 |
+
0.0,
|
| 15 |
+
-0.962890625,
|
| 16 |
+
-0.962890625,
|
| 17 |
+
-0.962890625,
|
| 18 |
+
-0.962890625,
|
| 19 |
+
-0.35498046875,
|
| 20 |
+
-0.417236328125,
|
| 21 |
+
9.7421875,
|
| 22 |
+
0.00868988037109375,
|
| 23 |
+
0.017425537109375,
|
| 24 |
+
-0.05926513671875,
|
| 25 |
+
8.234375,
|
| 26 |
+
0.0,
|
| 27 |
+
0.0,
|
| 28 |
+
5.734375,
|
| 29 |
+
-1.57421875
|
| 30 |
+
]
|
| 31 |
+
],
|
| 32 |
+
"frame": 2,
|
| 33 |
+
"imageSizes": [
|
| 34 |
+
[
|
| 35 |
+
900,
|
| 36 |
+
1600
|
| 37 |
+
],
|
| 38 |
+
[
|
| 39 |
+
900,
|
| 40 |
+
1600
|
| 41 |
+
],
|
| 42 |
+
[
|
| 43 |
+
900,
|
| 44 |
+
1600
|
| 45 |
+
],
|
| 46 |
+
[
|
| 47 |
+
900,
|
| 48 |
+
1600
|
| 49 |
+
],
|
| 50 |
+
[
|
| 51 |
+
900,
|
| 52 |
+
1600
|
| 53 |
+
],
|
| 54 |
+
[
|
| 55 |
+
900,
|
| 56 |
+
1600
|
| 57 |
+
]
|
| 58 |
+
],
|
| 59 |
+
"lidar2img": [
|
| 60 |
+
[
|
| 61 |
+
[
|
| 62 |
+
[
|
| 63 |
+
621.0,
|
| 64 |
+
420.75,
|
| 65 |
+
17.234375,
|
| 66 |
+
-177.625
|
| 67 |
+
],
|
| 68 |
+
[
|
| 69 |
+
-8.625,
|
| 70 |
+
268.5,
|
| 71 |
+
-612.5,
|
| 72 |
+
-322.75
|
| 73 |
+
],
|
| 74 |
+
[
|
| 75 |
+
-0.0125732421875,
|
| 76 |
+
0.99853515625,
|
| 77 |
+
0.054351806640625,
|
| 78 |
+
-0.4287109375
|
| 79 |
+
],
|
| 80 |
+
[
|
| 81 |
+
0.0,
|
| 82 |
+
0.0,
|
| 83 |
+
0.0,
|
| 84 |
+
1.0
|
| 85 |
+
]
|
| 86 |
+
],
|
| 87 |
+
[
|
| 88 |
+
[
|
| 89 |
+
682.5,
|
| 90 |
+
-309.25,
|
| 91 |
+
-19.671875,
|
| 92 |
+
-229.625
|
| 93 |
+
],
|
| 94 |
+
[
|
| 95 |
+
190.25,
|
| 96 |
+
160.5,
|
| 97 |
+
-619.5,
|
| 98 |
+
-345.5
|
| 99 |
+
],
|
| 100 |
+
[
|
| 101 |
+
0.84326171875,
|
| 102 |
+
0.53662109375,
|
| 103 |
+
0.033203125,
|
| 104 |
+
-0.60986328125
|
| 105 |
+
],
|
| 106 |
+
[
|
| 107 |
+
0.0,
|
| 108 |
+
0.0,
|
| 109 |
+
0.0,
|
| 110 |
+
1.0
|
| 111 |
+
]
|
| 112 |
+
],
|
| 113 |
+
[
|
| 114 |
+
[
|
| 115 |
+
15.3203125,
|
| 116 |
+
751.5,
|
| 117 |
+
39.03125,
|
| 118 |
+
-156.0
|
| 119 |
+
],
|
| 120 |
+
[
|
| 121 |
+
-194.0,
|
| 122 |
+
160.125,
|
| 123 |
+
-619.0,
|
| 124 |
+
-340.5
|
| 125 |
+
],
|
| 126 |
+
[
|
| 127 |
+
-0.82421875,
|
| 128 |
+
0.56494140625,
|
| 129 |
+
0.040283203125,
|
| 130 |
+
-0.53515625
|
| 131 |
+
],
|
| 132 |
+
[
|
| 133 |
+
0.0,
|
| 134 |
+
0.0,
|
| 135 |
+
0.0,
|
| 136 |
+
1.0
|
| 137 |
+
]
|
| 138 |
+
],
|
| 139 |
+
[
|
| 140 |
+
[
|
| 141 |
+
-402.0,
|
| 142 |
+
-425.5,
|
| 143 |
+
-13.5234375,
|
| 144 |
+
-436.5
|
| 145 |
+
],
|
| 146 |
+
[
|
| 147 |
+
-5.203125,
|
| 148 |
+
-222.5,
|
| 149 |
+
-407.5,
|
| 150 |
+
-354.75
|
| 151 |
+
],
|
| 152 |
+
[
|
| 153 |
+
-0.00809478759765625,
|
| 154 |
+
-0.9990234375,
|
| 155 |
+
-0.039459228515625,
|
| 156 |
+
-1.0185546875
|
| 157 |
+
],
|
| 158 |
+
[
|
| 159 |
+
0.0,
|
| 160 |
+
0.0,
|
| 161 |
+
0.0,
|
| 162 |
+
1.0
|
| 163 |
+
]
|
| 164 |
+
],
|
| 165 |
+
[
|
| 166 |
+
[
|
| 167 |
+
-593.5,
|
| 168 |
+
461.75,
|
| 169 |
+
26.640625,
|
| 170 |
+
-311.0
|
| 171 |
+
],
|
| 172 |
+
[
|
| 173 |
+
-231.25,
|
| 174 |
+
-51.1875,
|
| 175 |
+
-626.5,
|
| 176 |
+
-281.0
|
| 177 |
+
],
|
| 178 |
+
[
|
| 179 |
+
-0.94775390625,
|
| 180 |
+
-0.3193359375,
|
| 181 |
+
0.003086090087890625,
|
| 182 |
+
-0.43310546875
|
| 183 |
+
],
|
| 184 |
+
[
|
| 185 |
+
0.0,
|
| 186 |
+
0.0,
|
| 187 |
+
0.0,
|
| 188 |
+
1.0
|
| 189 |
+
]
|
| 190 |
+
],
|
| 191 |
+
[
|
| 192 |
+
[
|
| 193 |
+
142.875,
|
| 194 |
+
-734.5,
|
| 195 |
+
-30.015625,
|
| 196 |
+
-136.625
|
| 197 |
+
],
|
| 198 |
+
[
|
| 199 |
+
222.875,
|
| 200 |
+
-61.03125,
|
| 201 |
+
-625.0,
|
| 202 |
+
-293.5
|
| 203 |
+
],
|
| 204 |
+
[
|
| 205 |
+
0.92431640625,
|
| 206 |
+
-0.3818359375,
|
| 207 |
+
-0.003200531005859375,
|
| 208 |
+
-0.4638671875
|
| 209 |
+
],
|
| 210 |
+
[
|
| 211 |
+
0.0,
|
| 212 |
+
0.0,
|
| 213 |
+
0.0,
|
| 214 |
+
1.0
|
| 215 |
+
]
|
| 216 |
+
]
|
| 217 |
+
]
|
| 218 |
+
],
|
| 219 |
+
"rotationCanBus": [
|
| 220 |
+
[
|
| 221 |
+
3.628173828125,
|
| 222 |
+
-2.1719970703125,
|
| 223 |
+
0.0,
|
| 224 |
+
-0.9627518057823181,
|
| 225 |
+
-0.007396822329610586,
|
| 226 |
+
-0.00615654094144702,
|
| 227 |
+
0.2702152729034424,
|
| 228 |
+
-0.3548637330532074,
|
| 229 |
+
-0.417147159576416,
|
| 230 |
+
9.745100975036621,
|
| 231 |
+
0.008687585592269897,
|
| 232 |
+
0.017426401376724243,
|
| 233 |
+
-0.059267766773700714,
|
| 234 |
+
8.235058784484863,
|
| 235 |
+
0.0,
|
| 236 |
+
0.0,
|
| 237 |
+
5.7360124588012695,
|
| 238 |
+
-1.574005126953125
|
| 239 |
+
]
|
| 240 |
+
],
|
| 241 |
+
"sceneId": "fcbccedd61424f1b85dcbf8f897f9754",
|
| 242 |
+
"shift": [
|
| 243 |
+
[
|
| 244 |
+
-0.0003199577331542969,
|
| 245 |
+
0.041290283203125
|
| 246 |
+
]
|
| 247 |
+
],
|
| 248 |
+
"version": 1
|
| 249 |
+
}
|
targets/QCS8550/FP16/calibration/sample_003.json
ADDED
|
@@ -0,0 +1,249 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"cameraOrder": [
|
| 3 |
+
"front",
|
| 4 |
+
"front_right",
|
| 5 |
+
"front_left",
|
| 6 |
+
"back",
|
| 7 |
+
"back_left",
|
| 8 |
+
"back_right"
|
| 9 |
+
],
|
| 10 |
+
"canBus": [
|
| 11 |
+
[
|
| 12 |
+
3.525390625,
|
| 13 |
+
-2.236328125,
|
| 14 |
+
0.0,
|
| 15 |
+
-0.95849609375,
|
| 16 |
+
-0.95849609375,
|
| 17 |
+
-0.95849609375,
|
| 18 |
+
-0.95849609375,
|
| 19 |
+
-0.257080078125,
|
| 20 |
+
-0.45556640625,
|
| 21 |
+
9.4375,
|
| 22 |
+
0.00681304931640625,
|
| 23 |
+
0.01085662841796875,
|
| 24 |
+
-0.058349609375,
|
| 25 |
+
8.1875,
|
| 26 |
+
0.0,
|
| 27 |
+
0.0,
|
| 28 |
+
5.70703125,
|
| 29 |
+
-1.673828125
|
| 30 |
+
]
|
| 31 |
+
],
|
| 32 |
+
"frame": 3,
|
| 33 |
+
"imageSizes": [
|
| 34 |
+
[
|
| 35 |
+
900,
|
| 36 |
+
1600
|
| 37 |
+
],
|
| 38 |
+
[
|
| 39 |
+
900,
|
| 40 |
+
1600
|
| 41 |
+
],
|
| 42 |
+
[
|
| 43 |
+
900,
|
| 44 |
+
1600
|
| 45 |
+
],
|
| 46 |
+
[
|
| 47 |
+
900,
|
| 48 |
+
1600
|
| 49 |
+
],
|
| 50 |
+
[
|
| 51 |
+
900,
|
| 52 |
+
1600
|
| 53 |
+
],
|
| 54 |
+
[
|
| 55 |
+
900,
|
| 56 |
+
1600
|
| 57 |
+
]
|
| 58 |
+
],
|
| 59 |
+
"lidar2img": [
|
| 60 |
+
[
|
| 61 |
+
[
|
| 62 |
+
[
|
| 63 |
+
621.0,
|
| 64 |
+
421.25,
|
| 65 |
+
17.484375,
|
| 66 |
+
-179.375
|
| 67 |
+
],
|
| 68 |
+
[
|
| 69 |
+
-8.7421875,
|
| 70 |
+
268.75,
|
| 71 |
+
-612.5,
|
| 72 |
+
-326.75
|
| 73 |
+
],
|
| 74 |
+
[
|
| 75 |
+
-0.01325225830078125,
|
| 76 |
+
0.99853515625,
|
| 77 |
+
0.054779052734375,
|
| 78 |
+
-0.435546875
|
| 79 |
+
],
|
| 80 |
+
[
|
| 81 |
+
0.0,
|
| 82 |
+
0.0,
|
| 83 |
+
0.0,
|
| 84 |
+
1.0
|
| 85 |
+
]
|
| 86 |
+
],
|
| 87 |
+
[
|
| 88 |
+
[
|
| 89 |
+
683.0,
|
| 90 |
+
-308.75,
|
| 91 |
+
-19.71875,
|
| 92 |
+
-226.75
|
| 93 |
+
],
|
| 94 |
+
[
|
| 95 |
+
190.125,
|
| 96 |
+
160.875,
|
| 97 |
+
-619.5,
|
| 98 |
+
-348.25
|
| 99 |
+
],
|
| 100 |
+
[
|
| 101 |
+
0.8427734375,
|
| 102 |
+
0.537109375,
|
| 103 |
+
0.033447265625,
|
| 104 |
+
-0.6123046875
|
| 105 |
+
],
|
| 106 |
+
[
|
| 107 |
+
0.0,
|
| 108 |
+
0.0,
|
| 109 |
+
0.0,
|
| 110 |
+
1.0
|
| 111 |
+
]
|
| 112 |
+
],
|
| 113 |
+
[
|
| 114 |
+
[
|
| 115 |
+
14.7265625,
|
| 116 |
+
751.5,
|
| 117 |
+
39.4375,
|
| 118 |
+
-161.125
|
| 119 |
+
],
|
| 120 |
+
[
|
| 121 |
+
-194.125,
|
| 122 |
+
160.375,
|
| 123 |
+
-619.0,
|
| 124 |
+
-344.75
|
| 125 |
+
],
|
| 126 |
+
[
|
| 127 |
+
-0.82470703125,
|
| 128 |
+
0.564453125,
|
| 129 |
+
0.04046630859375,
|
| 130 |
+
-0.54052734375
|
| 131 |
+
],
|
| 132 |
+
[
|
| 133 |
+
0.0,
|
| 134 |
+
0.0,
|
| 135 |
+
0.0,
|
| 136 |
+
1.0
|
| 137 |
+
]
|
| 138 |
+
],
|
| 139 |
+
[
|
| 140 |
+
[
|
| 141 |
+
-401.75,
|
| 142 |
+
-425.5,
|
| 143 |
+
-13.515625,
|
| 144 |
+
-435.0
|
| 145 |
+
],
|
| 146 |
+
[
|
| 147 |
+
-5.16015625,
|
| 148 |
+
-222.5,
|
| 149 |
+
-407.5,
|
| 150 |
+
-354.25
|
| 151 |
+
],
|
| 152 |
+
[
|
| 153 |
+
-0.007904052734375,
|
| 154 |
+
-0.9990234375,
|
| 155 |
+
-0.0394287109375,
|
| 156 |
+
-1.0146484375
|
| 157 |
+
],
|
| 158 |
+
[
|
| 159 |
+
0.0,
|
| 160 |
+
0.0,
|
| 161 |
+
0.0,
|
| 162 |
+
1.0
|
| 163 |
+
]
|
| 164 |
+
],
|
| 165 |
+
[
|
| 166 |
+
[
|
| 167 |
+
-593.5,
|
| 168 |
+
461.5,
|
| 169 |
+
26.625,
|
| 170 |
+
-312.0
|
| 171 |
+
],
|
| 172 |
+
[
|
| 173 |
+
-231.25,
|
| 174 |
+
-51.25,
|
| 175 |
+
-626.5,
|
| 176 |
+
-281.0
|
| 177 |
+
],
|
| 178 |
+
[
|
| 179 |
+
-0.94775390625,
|
| 180 |
+
-0.319580078125,
|
| 181 |
+
0.0030803680419921875,
|
| 182 |
+
-0.433349609375
|
| 183 |
+
],
|
| 184 |
+
[
|
| 185 |
+
0.0,
|
| 186 |
+
0.0,
|
| 187 |
+
0.0,
|
| 188 |
+
1.0
|
| 189 |
+
]
|
| 190 |
+
],
|
| 191 |
+
[
|
| 192 |
+
[
|
| 193 |
+
143.25,
|
| 194 |
+
-734.5,
|
| 195 |
+
-30.078125,
|
| 196 |
+
-132.375
|
| 197 |
+
],
|
| 198 |
+
[
|
| 199 |
+
222.875,
|
| 200 |
+
-60.875,
|
| 201 |
+
-625.0,
|
| 202 |
+
-294.25
|
| 203 |
+
],
|
| 204 |
+
[
|
| 205 |
+
0.92431640625,
|
| 206 |
+
-0.381591796875,
|
| 207 |
+
-0.003204345703125,
|
| 208 |
+
-0.4609375
|
| 209 |
+
],
|
| 210 |
+
[
|
| 211 |
+
0.0,
|
| 212 |
+
0.0,
|
| 213 |
+
0.0,
|
| 214 |
+
1.0
|
| 215 |
+
]
|
| 216 |
+
]
|
| 217 |
+
]
|
| 218 |
+
],
|
| 219 |
+
"rotationCanBus": [
|
| 220 |
+
[
|
| 221 |
+
3.52557373046875,
|
| 222 |
+
-2.235595703125,
|
| 223 |
+
0.0,
|
| 224 |
+
-0.9586126804351807,
|
| 225 |
+
-0.00886558648198843,
|
| 226 |
+
-0.013591880910098553,
|
| 227 |
+
0.2842506468296051,
|
| 228 |
+
-0.25705933570861816,
|
| 229 |
+
-0.45562514662742615,
|
| 230 |
+
9.434834480285645,
|
| 231 |
+
0.006814147345721722,
|
| 232 |
+
0.010853799059987068,
|
| 233 |
+
-0.05835989862680435,
|
| 234 |
+
8.190821647644043,
|
| 235 |
+
0.0,
|
| 236 |
+
0.0,
|
| 237 |
+
5.706801891326904,
|
| 238 |
+
-1.67364501953125
|
| 239 |
+
]
|
| 240 |
+
],
|
| 241 |
+
"sceneId": "fcbccedd61424f1b85dcbf8f897f9754",
|
| 242 |
+
"shift": [
|
| 243 |
+
[
|
| 244 |
+
-0.0004591941833496094,
|
| 245 |
+
0.040771484375
|
| 246 |
+
]
|
| 247 |
+
],
|
| 248 |
+
"version": 1
|
| 249 |
+
}
|
targets/QCS8550/FP16/code/.gitkeep
ADDED
|
File without changes
|
targets/QCS8550/FP16/code/README.md
ADDED
|
@@ -0,0 +1,150 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
## Model Information
|
| 2 |
+
|
| 3 |
+
### Source model
|
| 4 |
+
- Input shape: [[1,6,3,480,800]]
|
| 5 |
+
- Number of parameters: 33.7M
|
| 6 |
+
- Model size: 128.5 FP32
|
| 7 |
+
- Output shape: boxes [[300,9]], scores [[300]], labels [[300]]
|
| 8 |
+
|
| 9 |
+
Source model repository: [BEVFormer](https://github.com/fundamentalvision/bevformer)
|
| 10 |
+
|
| 11 |
+
## Inference with AidLite SDK
|
| 12 |
+
|
| 13 |
+
### SDK installation
|
| 14 |
+
Model Farm uses AidLite SDK as the model inference SDK. For details, please refer to the [AidLite Developer Documentation](https://docs.aidlux.com/software/ai-sdk/aidlite_guide)
|
| 15 |
+
|
| 16 |
+
```bash
|
| 17 |
+
sudo aid-pkg update
|
| 18 |
+
sudo aid-pkg install aidlite-sdk
|
| 19 |
+
sudo aid-pkg install aidlite-{QNN VERSION}
|
| 20 |
+
```
|
| 21 |
+
|
| 22 |
+
Verify AidLite SDK:
|
| 23 |
+
|
| 24 |
+
```bash
|
| 25 |
+
python3 -c "import aidlite ; print(aidlite.get_library_version())"
|
| 26 |
+
python3 -c "import aidlite ; print(aidlite.get_py_library_version())"
|
| 27 |
+
```
|
| 28 |
+
|
| 29 |
+
### Environment Dependencies
|
| 30 |
+
|
| 31 |
+
Except for AidLite SDK, this BEVFormer demo also requires the following board-side Python environment.
|
| 32 |
+
|
| 33 |
+
Verified environment:
|
| 34 |
+
|
| 35 |
+
```bash
|
| 36 |
+
Python 3.10.12
|
| 37 |
+
numpy 1.26.4
|
| 38 |
+
opencv-python / cv2 4.13.0
|
| 39 |
+
Pillow / PIL 10.4.0
|
| 40 |
+
```
|
| 41 |
+
|
| 42 |
+
Required Python packages:
|
| 43 |
+
|
| 44 |
+
```bash
|
| 45 |
+
numpy
|
| 46 |
+
opencv-python
|
| 47 |
+
Pillow
|
| 48 |
+
```
|
| 49 |
+
|
| 50 |
+
Quick check:
|
| 51 |
+
|
| 52 |
+
```bash
|
| 53 |
+
python3 -c "import numpy, cv2; from PIL import Image; print('python env ok')"
|
| 54 |
+
```
|
| 55 |
+
|
| 56 |
+
The demo supports model directories for QCS8550, IQ9, QCS8625, and IQ8. A normal host or development container can only run dry-run package checks and cannot execute real QNN inference without AidLite.
|
| 57 |
+
|
| 58 |
+
## Model Directory Rule
|
| 59 |
+
|
| 60 |
+
Model SHA verification is always enabled. The platform is inferred from the model path, so no extra SHA option is required.
|
| 61 |
+
|
| 62 |
+
Supported model directories:
|
| 63 |
+
|
| 64 |
+
```bash
|
| 65 |
+
./models/QCS8550/FP16
|
| 66 |
+
./models/IQ9/FP16
|
| 67 |
+
./models/QCS8625/FP16
|
| 68 |
+
./models/IQ8/FP16
|
| 69 |
+
```
|
| 70 |
+
|
| 71 |
+
Platform mapping:
|
| 72 |
+
|
| 73 |
+
```text
|
| 74 |
+
QCS8550 -> QCS8550 model SHA table
|
| 75 |
+
IQ9 -> QCS8550 model SHA table
|
| 76 |
+
QCS8625 -> QCS8625 model SHA table
|
| 77 |
+
IQ8 -> QCS8625 model SHA table
|
| 78 |
+
```
|
| 79 |
+
|
| 80 |
+
Each platform directory must contain all four context files:
|
| 81 |
+
|
| 82 |
+
```bash
|
| 83 |
+
backbone_context.bin.aidem
|
| 84 |
+
scene_start_encoder_context.bin.aidem
|
| 85 |
+
temporal_encoder_context.bin.aidem
|
| 86 |
+
decoder_context.bin.aidem
|
| 87 |
+
```
|
| 88 |
+
|
| 89 |
+
If all four `--*_model` arguments are provided, the model paths decide the platform. All four paths must belong to the same platform directory. If model arguments are omitted, the script uses the first complete directory found in this order:
|
| 90 |
+
|
| 91 |
+
```text
|
| 92 |
+
QCS8550, IQ9, QCS8625, IQ8
|
| 93 |
+
```
|
| 94 |
+
|
| 95 |
+
## Run Demo
|
| 96 |
+
|
| 97 |
+
Basic command:
|
| 98 |
+
|
| 99 |
+
```bash
|
| 100 |
+
cd /home/aidlux/bevformer_delivery_demo_2026_07_08
|
| 101 |
+
python3 code/python/run_test.py --invoke_nums 4 --output_dir ./outputs/final_sample4
|
| 102 |
+
```
|
| 103 |
+
|
| 104 |
+
Full command with explicit model paths and dataset manifest:
|
| 105 |
+
|
| 106 |
+
```bash
|
| 107 |
+
python3 code/python/run_test.py \
|
| 108 |
+
--backbone_model ./models/QCS8625/FP16/backbone_context.bin.aidem \
|
| 109 |
+
--scene_start_encoder_model ./models/QCS8625/FP16/scene_start_encoder_context.bin.aidem \
|
| 110 |
+
--encoder_model ./models/QCS8625/FP16/temporal_encoder_context.bin.aidem \
|
| 111 |
+
--decoder_model ./models/QCS8625/FP16/decoder_context.bin.aidem \
|
| 112 |
+
--asset_manifest ./code/python/datasets/sample4/asset_manifest.json \
|
| 113 |
+
--invoke_nums 4 \
|
| 114 |
+
--output_dir ./outputs/final_sample4_qcs8625
|
| 115 |
+
```
|
| 116 |
+
|
| 117 |
+
The basic command auto-discovers the first complete model directory under `models/<platform>/FP16`. For QCS8550, IQ9, or IQ8, replace `QCS8625` in the full command with `QCS8550`, `IQ9`, or `IQ8`. The four model paths must use the same platform directory.
|
| 118 |
+
|
| 119 |
+
Dry-run package check:
|
| 120 |
+
|
| 121 |
+
```bash
|
| 122 |
+
python3 code/python/run_test.py --dry_run --check_raw_assets
|
| 123 |
+
```
|
| 124 |
+
|
| 125 |
+
The demo input is specified by:
|
| 126 |
+
|
| 127 |
+
```bash
|
| 128 |
+
./code/python/datasets/sample4/asset_manifest.json
|
| 129 |
+
```
|
| 130 |
+
|
| 131 |
+
The manifest describes four continuous BEVFormer frames. Each frame contains six raw camera JPG images and the required calibration / temporal auxiliary tensors. The demo starts from original six-camera images, not preprocessed image tensors.
|
| 132 |
+
|
| 133 |
+
Frame execution order:
|
| 134 |
+
|
| 135 |
+
```bash
|
| 136 |
+
frame 000: scene-start encoder
|
| 137 |
+
frame 001: temporal encoder
|
| 138 |
+
frame 002: temporal encoder
|
| 139 |
+
frame 003: temporal encoder
|
| 140 |
+
```
|
| 141 |
+
|
| 142 |
+
Main output files include:
|
| 143 |
+
|
| 144 |
+
```bash
|
| 145 |
+
bevformer_demo_summary.json
|
| 146 |
+
run.log
|
| 147 |
+
frameXXX_final_coordinates.npz
|
| 148 |
+
frameXXX_camera_grid.png
|
| 149 |
+
sample4_camera_grid.gif
|
| 150 |
+
```
|
targets/QCS8550/FP16/code/python/bevformer.py
ADDED
|
@@ -0,0 +1,611 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
from __future__ import annotations
|
| 2 |
+
|
| 3 |
+
import time
|
| 4 |
+
from pathlib import Path
|
| 5 |
+
from typing import Any
|
| 6 |
+
|
| 7 |
+
import numpy as np
|
| 8 |
+
|
| 9 |
+
import portable_numpy_nmsfreecoder as portable_nms
|
| 10 |
+
from camera_grid_visualization import save_camera_grid_gif, save_camera_grid_summary, save_camera_grid_visualization
|
| 11 |
+
from utils import (
|
| 12 |
+
EXPECTED_TENSORS,
|
| 13 |
+
SHAPES,
|
| 14 |
+
as_encoder_img_feat,
|
| 15 |
+
elapsed_ms,
|
| 16 |
+
load_backbone_images,
|
| 17 |
+
load_json,
|
| 18 |
+
load_record,
|
| 19 |
+
normalize_rc,
|
| 20 |
+
rotate_prev_bev,
|
| 21 |
+
save_final_coordinates,
|
| 22 |
+
save_raw_outputs,
|
| 23 |
+
sha256_file,
|
| 24 |
+
stats,
|
| 25 |
+
)
|
| 26 |
+
|
| 27 |
+
|
| 28 |
+
QCS8550_SHA256 = {
|
| 29 |
+
"backbone": "9a7a5137a2ed3072cdc2c567822a2316b9501a591ffa204b17568f6c80d8844e",
|
| 30 |
+
"encoder_temporal": "25f0063fc7d1b125522d3ac9c53b26a9e270d504fc6c908ae5737ec78ad65b94",
|
| 31 |
+
"encoder_scene_start": "eed541fa4138a177432e2290b4131799d88534005789bef9962e2bbb9e008186",
|
| 32 |
+
"decoder": "f6888090751bf1ff6ecbaef966b3b1a6c97793784f921001b493893a8ce6196c",
|
| 33 |
+
}
|
| 34 |
+
|
| 35 |
+
IQ9_SHA256 = {
|
| 36 |
+
"backbone": "54c042b09eff47d99998762a375fe606bc3a83350ebf1085399384f50fe2dbfb",
|
| 37 |
+
"encoder_temporal": "b376fe6112794278a4b3f787b34e387d62cd81de00312021ca927448c4e9c253",
|
| 38 |
+
"encoder_scene_start": "8b0987fe7a21a79a4bc5975f3732b7d9c60611fadc813a96e438dfc85a59fb2e",
|
| 39 |
+
"decoder": "60952eaa1d35c648d5992a4f8ef2563410b8b6fe68e8ad6095f1231921ea0ff5",
|
| 40 |
+
}
|
| 41 |
+
|
| 42 |
+
QCS8625_SHA256 = {
|
| 43 |
+
"backbone": "552bb2f456638c19bfcd88d79e5c15a671fddca21077332c6e0cb1d1cd2641de",
|
| 44 |
+
"encoder_temporal": "046652a264dce3728b11433b1a3378e6c4b894e1286d007d940daec4ec5a2608",
|
| 45 |
+
"encoder_scene_start": "7071b3937d79191412db50688136f88f70c917a8bff6c0e8af649157555be592",
|
| 46 |
+
"decoder": "be322fead0610e0321ce165cc0c1d2686768b6c6e1b3241c8ef89f45f6d2bbc5",
|
| 47 |
+
}
|
| 48 |
+
|
| 49 |
+
IQ8_SHA256 = {
|
| 50 |
+
"backbone": "e7f75ca4ca7fda12ecdfbb14694a906165b70ecdc8e605b54174d82a042f25b1",
|
| 51 |
+
"encoder_temporal": "f79e2c208fe63eec8f30bf289310dcc429cf6b87a05dd3397aeda6d08aabaf74",
|
| 52 |
+
"encoder_scene_start": "62c6b62139525d238c447d65f47004f65bd9e2cd88beae5b580bdd7d80687f81",
|
| 53 |
+
"decoder": "814534559bf5ffd00f4316302b7804434e19a2420117ec9b5f46fe5dbce887f1",
|
| 54 |
+
}
|
| 55 |
+
|
| 56 |
+
MODEL_SHA256 = {
|
| 57 |
+
"QCS8550": QCS8550_SHA256,
|
| 58 |
+
"IQ9": IQ9_SHA256,
|
| 59 |
+
"QCS8625": QCS8625_SHA256,
|
| 60 |
+
"IQ8": IQ8_SHA256,
|
| 61 |
+
}
|
| 62 |
+
|
| 63 |
+
|
| 64 |
+
MODEL_NAMES = (
|
| 65 |
+
"backbone",
|
| 66 |
+
"encoder_temporal",
|
| 67 |
+
"encoder_scene_start",
|
| 68 |
+
"decoder",
|
| 69 |
+
)
|
| 70 |
+
|
| 71 |
+
NUSCENES_CLASSES = (
|
| 72 |
+
"car",
|
| 73 |
+
"truck",
|
| 74 |
+
"construction_vehicle",
|
| 75 |
+
"bus",
|
| 76 |
+
"trailer",
|
| 77 |
+
"barrier",
|
| 78 |
+
"motorcycle",
|
| 79 |
+
"bicycle",
|
| 80 |
+
"pedestrian",
|
| 81 |
+
"traffic_cone",
|
| 82 |
+
)
|
| 83 |
+
|
| 84 |
+
|
| 85 |
+
class BevFormerModel:
|
| 86 |
+
def __init__(
|
| 87 |
+
self,
|
| 88 |
+
backbone_model: str,
|
| 89 |
+
encoder_temporal_model: str,
|
| 90 |
+
encoder_scene_start_model: str,
|
| 91 |
+
decoder_model: str,
|
| 92 |
+
model_type: str = "QNN248",
|
| 93 |
+
expected_sha256: dict[str, str] | None = None,
|
| 94 |
+
):
|
| 95 |
+
try:
|
| 96 |
+
import aidlite
|
| 97 |
+
except ModuleNotFoundError as exc:
|
| 98 |
+
raise RuntimeError(
|
| 99 |
+
"AidLite Python runtime is not available in this environment. "
|
| 100 |
+
"Use --dry_run to demonstrate the package structure in a normal container, "
|
| 101 |
+
"or run without --dry_run on the board / Container B where "
|
| 102 |
+
"`python3 -c \"import aidlite\"` succeeds."
|
| 103 |
+
) from exc
|
| 104 |
+
|
| 105 |
+
if model_type.upper() != "QNN248":
|
| 106 |
+
raise ValueError("This demo is pinned to QNN248 contexts")
|
| 107 |
+
if (
|
| 108 |
+
int(aidlite.FrameworkType.TYPE_QNN248),
|
| 109 |
+
int(aidlite.ImplementType.TYPE_LOCAL),
|
| 110 |
+
int(aidlite.AccelerateType.TYPE_DSP),
|
| 111 |
+
) != (110, 3, 3):
|
| 112 |
+
raise RuntimeError("AidLite enum contract mismatch")
|
| 113 |
+
|
| 114 |
+
self.aidlite = aidlite
|
| 115 |
+
model_paths = {
|
| 116 |
+
"backbone": backbone_model,
|
| 117 |
+
"encoder_temporal": encoder_temporal_model,
|
| 118 |
+
"encoder_scene_start": encoder_scene_start_model,
|
| 119 |
+
"decoder": decoder_model,
|
| 120 |
+
}
|
| 121 |
+
self.model_platform = self.resolve_model_platform(model_paths)
|
| 122 |
+
self.expected_sha256 = expected_sha256 or MODEL_SHA256[self.model_platform]
|
| 123 |
+
self.interpreters: dict[str, Any] = {}
|
| 124 |
+
self.model_records: dict[str, Any] = {}
|
| 125 |
+
self.model_load_timing_ms: dict[str, float] = {}
|
| 126 |
+
model_load_start = time.perf_counter_ns()
|
| 127 |
+
|
| 128 |
+
for name in MODEL_NAMES:
|
| 129 |
+
path = model_paths[name]
|
| 130 |
+
load_start = time.perf_counter_ns()
|
| 131 |
+
interpreter, record = self._create_loaded_interpreter(name, str(path))
|
| 132 |
+
self.model_load_timing_ms[name] = elapsed_ms(load_start)
|
| 133 |
+
self.interpreters[name] = interpreter
|
| 134 |
+
self.model_records[name] = record
|
| 135 |
+
|
| 136 |
+
self.model_load_timing_ms["total"] = elapsed_ms(model_load_start)
|
| 137 |
+
|
| 138 |
+
def __del__(self):
|
| 139 |
+
for interpreter in reversed(list(getattr(self, "interpreters", {}).values())):
|
| 140 |
+
for method_name in ("destroy", "destory"):
|
| 141 |
+
if hasattr(interpreter, method_name):
|
| 142 |
+
try:
|
| 143 |
+
getattr(interpreter, method_name)()
|
| 144 |
+
except Exception:
|
| 145 |
+
pass
|
| 146 |
+
break
|
| 147 |
+
|
| 148 |
+
def _create_model(self, model_path: str) -> Any:
|
| 149 |
+
try:
|
| 150 |
+
return self.aidlite.Model.create_instance(model_path=model_path)
|
| 151 |
+
except TypeError:
|
| 152 |
+
return self.aidlite.Model.create_instance(model_path)
|
| 153 |
+
|
| 154 |
+
def _build_interpreter(self, model: Any, config: Any) -> Any:
|
| 155 |
+
for method_name in ("build_interpreter_from_model_and_config", "build_interpretper_from_model_and_config"):
|
| 156 |
+
if hasattr(self.aidlite.InterpreterBuilder, method_name):
|
| 157 |
+
method = getattr(self.aidlite.InterpreterBuilder, method_name)
|
| 158 |
+
try:
|
| 159 |
+
return method(model=model, config=config)
|
| 160 |
+
except TypeError:
|
| 161 |
+
return method(model, config)
|
| 162 |
+
raise RuntimeError("No supported AidLite InterpreterBuilder method")
|
| 163 |
+
|
| 164 |
+
@staticmethod
|
| 165 |
+
def infer_model_platform(model_path: str | Path) -> str:
|
| 166 |
+
parts = [part.upper() for part in Path(model_path).expanduser().parts]
|
| 167 |
+
matches = [platform for platform in MODEL_SHA256 if platform.upper() in parts]
|
| 168 |
+
if len(matches) != 1:
|
| 169 |
+
supported = ", ".join(MODEL_SHA256)
|
| 170 |
+
raise RuntimeError(f"Cannot infer model platform from path={model_path}. Expected one of: {supported}")
|
| 171 |
+
return matches[0]
|
| 172 |
+
|
| 173 |
+
@classmethod
|
| 174 |
+
def resolve_model_platform(cls, model_paths: dict[str, str | Path]) -> str:
|
| 175 |
+
platforms = {name: cls.infer_model_platform(path) for name, path in model_paths.items()}
|
| 176 |
+
unique = set(platforms.values())
|
| 177 |
+
if len(unique) != 1:
|
| 178 |
+
raise RuntimeError(f"Model platform mismatch: {platforms}")
|
| 179 |
+
return next(iter(unique))
|
| 180 |
+
|
| 181 |
+
@staticmethod
|
| 182 |
+
def _flatten_tensor_info(groups: Any) -> list[dict[str, Any]]:
|
| 183 |
+
records: list[dict[str, Any]] = []
|
| 184 |
+
if groups is None:
|
| 185 |
+
return records
|
| 186 |
+
for graph_index, group in enumerate(groups):
|
| 187 |
+
try:
|
| 188 |
+
tensors = list(group)
|
| 189 |
+
except TypeError:
|
| 190 |
+
tensors = [group]
|
| 191 |
+
for tensor_index, info in enumerate(tensors):
|
| 192 |
+
records.append({
|
| 193 |
+
"graph_index": graph_index,
|
| 194 |
+
"tensor_index": tensor_index,
|
| 195 |
+
"name": str(getattr(info, "name", "")),
|
| 196 |
+
"element_count": int(getattr(info, "element_count", -1)),
|
| 197 |
+
"shape": [int(v) for v in getattr(info, "shape", [])],
|
| 198 |
+
"element_type": str(getattr(info, "element_type", "")),
|
| 199 |
+
})
|
| 200 |
+
return records
|
| 201 |
+
|
| 202 |
+
def _create_loaded_interpreter(self, name: str, model_path: str) -> tuple[Any, dict[str, Any]]:
|
| 203 |
+
path = Path(model_path).expanduser().resolve()
|
| 204 |
+
if not path.is_file():
|
| 205 |
+
raise FileNotFoundError(path)
|
| 206 |
+
actual_sha = sha256_file(path)
|
| 207 |
+
expected_sha = self.expected_sha256[name]
|
| 208 |
+
if actual_sha != expected_sha:
|
| 209 |
+
raise RuntimeError(
|
| 210 |
+
f"{name} context SHA mismatch for {self.model_platform}: "
|
| 211 |
+
f"expected={expected_sha} actual={actual_sha}"
|
| 212 |
+
)
|
| 213 |
+
|
| 214 |
+
model = self._create_model(str(path))
|
| 215 |
+
config = self.aidlite.Config.create_instance()
|
| 216 |
+
if model is None or config is None:
|
| 217 |
+
raise RuntimeError(f"{name}: Model/Config creation failed")
|
| 218 |
+
config.framework_type = self.aidlite.FrameworkType.TYPE_QNN248
|
| 219 |
+
config.implement_type = self.aidlite.ImplementType.TYPE_LOCAL
|
| 220 |
+
config.accelerate_type = self.aidlite.AccelerateType.TYPE_DSP
|
| 221 |
+
config.qnn_shared_buffer = 0
|
| 222 |
+
|
| 223 |
+
interpreter = self._build_interpreter(model, config)
|
| 224 |
+
if interpreter is None:
|
| 225 |
+
raise RuntimeError(f"{name}: interpreter creation failed")
|
| 226 |
+
init_rc = normalize_rc(interpreter.init())
|
| 227 |
+
load_rc = normalize_rc(interpreter.load_model())
|
| 228 |
+
if init_rc != 0 or load_rc != 0:
|
| 229 |
+
raise RuntimeError(f"{name}: init/load failed init={init_rc} load={load_rc}")
|
| 230 |
+
|
| 231 |
+
inputs = self._flatten_tensor_info(interpreter.get_input_tensor_info())
|
| 232 |
+
outputs = self._flatten_tensor_info(interpreter.get_output_tensor_info())
|
| 233 |
+
actual_inputs = {item["name"]: item["element_count"] for item in inputs}
|
| 234 |
+
actual_outputs = {item["name"]: item["element_count"] for item in outputs}
|
| 235 |
+
expected = EXPECTED_TENSORS[name]
|
| 236 |
+
if actual_inputs != expected["inputs"] or actual_outputs != expected["outputs"]:
|
| 237 |
+
raise RuntimeError(f"{name}: tensor contract mismatch inputs={actual_inputs} outputs={actual_outputs}")
|
| 238 |
+
|
| 239 |
+
return interpreter, {
|
| 240 |
+
"name": name,
|
| 241 |
+
"platform": self.model_platform,
|
| 242 |
+
"path": str(path),
|
| 243 |
+
"sha256": actual_sha,
|
| 244 |
+
"inputs": inputs,
|
| 245 |
+
"outputs": outputs,
|
| 246 |
+
}
|
| 247 |
+
|
| 248 |
+
def _set_input(self, interpreter: Any, name: str, value: np.ndarray) -> float:
|
| 249 |
+
tensor = np.ascontiguousarray(value, dtype=np.float32)
|
| 250 |
+
start = time.perf_counter_ns()
|
| 251 |
+
rc = normalize_rc(interpreter.set_input_tensor(in_tensor_tag=name, input_data=tensor))
|
| 252 |
+
duration = elapsed_ms(start)
|
| 253 |
+
if rc != 0:
|
| 254 |
+
raise RuntimeError(f"set_input_tensor failed name={name} rc={rc}")
|
| 255 |
+
return duration
|
| 256 |
+
|
| 257 |
+
def _invoke(self, interpreter: Any, name: str) -> float:
|
| 258 |
+
start = time.perf_counter_ns()
|
| 259 |
+
rc = normalize_rc(interpreter.invoke())
|
| 260 |
+
duration = elapsed_ms(start)
|
| 261 |
+
if rc != 0:
|
| 262 |
+
raise RuntimeError(f"{name}: invoke failed rc={rc}")
|
| 263 |
+
return duration
|
| 264 |
+
|
| 265 |
+
def _get_output(self, interpreter: Any, name: str, shape: tuple[int, ...]) -> tuple[np.ndarray, float]:
|
| 266 |
+
start = time.perf_counter_ns()
|
| 267 |
+
value = interpreter.get_output_tensor(out_tensor_tag=name)
|
| 268 |
+
duration = elapsed_ms(start)
|
| 269 |
+
if value is None:
|
| 270 |
+
raise RuntimeError(f"get_output_tensor returned None name={name}")
|
| 271 |
+
array = np.asarray(value, dtype=np.float32).reshape(shape)
|
| 272 |
+
if not np.isfinite(array).all():
|
| 273 |
+
raise RuntimeError(f"{name}: non-finite output")
|
| 274 |
+
return np.ascontiguousarray(array, dtype=np.float32), duration
|
| 275 |
+
|
| 276 |
+
def run_frame(
|
| 277 |
+
self,
|
| 278 |
+
frame_index: int,
|
| 279 |
+
frame_manifest: dict[str, Any],
|
| 280 |
+
repo_root: str | Path,
|
| 281 |
+
previous_frame_bev_embed: np.ndarray | None,
|
| 282 |
+
check_image_sha: bool = False,
|
| 283 |
+
preprocess_workers: int = 6,
|
| 284 |
+
) -> tuple[np.ndarray, np.ndarray, np.ndarray, dict[str, Any]]:
|
| 285 |
+
"""Run one BEVFormer frame on the board.
|
| 286 |
+
|
| 287 |
+
The QNN context tensor names still follow the exported model contract
|
| 288 |
+
such as ``can_bus``, ``lidar2img`` and ``prev_bev``. The Python
|
| 289 |
+
variable names below describe their deployment meaning more explicitly:
|
| 290 |
+
|
| 291 |
+
- scene-start frame: use the scene-start encoder; no previous BEV state.
|
| 292 |
+
- temporal frame: rotate the previous frame BEV state, then feed it as
|
| 293 |
+
the current frame ``prev_bev`` tensor.
|
| 294 |
+
"""
|
| 295 |
+
|
| 296 |
+
frame_start = time.perf_counter_ns()
|
| 297 |
+
is_scene_start_frame = bool(frame_manifest.get("is_scene_start", False))
|
| 298 |
+
asset_records = frame_manifest["assets"]
|
| 299 |
+
timing: dict[str, float] = {}
|
| 300 |
+
|
| 301 |
+
preprocess_start = time.perf_counter_ns()
|
| 302 |
+
padded_camera_tensor, image_preprocess = load_backbone_images(
|
| 303 |
+
asset_records,
|
| 304 |
+
repo_root,
|
| 305 |
+
check_image_sha=check_image_sha,
|
| 306 |
+
preprocess_workers=preprocess_workers,
|
| 307 |
+
)
|
| 308 |
+
timing["image_preprocess_ms"] = elapsed_ms(preprocess_start)
|
| 309 |
+
|
| 310 |
+
ego_motion_can_bus = load_record(asset_records["can_bus"], repo_root).astype(np.float32)
|
| 311 |
+
camera_lidar2img_matrices = load_record(asset_records["lidar2img"], repo_root).astype(np.float32)
|
| 312 |
+
|
| 313 |
+
timing["backbone_set_input_ms"] = self._set_input(
|
| 314 |
+
self.interpreters["backbone"],
|
| 315 |
+
"images",
|
| 316 |
+
padded_camera_tensor,
|
| 317 |
+
)
|
| 318 |
+
timing["backbone_invoke_ms"] = self._invoke(self.interpreters["backbone"], "backbone")
|
| 319 |
+
backbone_image_features, timing["backbone_get_output_ms"] = self._get_output(
|
| 320 |
+
self.interpreters["backbone"], "img_feat", SHAPES["img_feat"]
|
| 321 |
+
)
|
| 322 |
+
encoder_image_features = as_encoder_img_feat(backbone_image_features)
|
| 323 |
+
|
| 324 |
+
if is_scene_start_frame:
|
| 325 |
+
selected_encoder_name = "encoder_scene_start"
|
| 326 |
+
selected_encoder_route = "scene_start_reset_prev_bev"
|
| 327 |
+
selected_encoder = self.interpreters[selected_encoder_name]
|
| 328 |
+
timing["encoder_set_input_ms"] = 0.0
|
| 329 |
+
scene_start_inputs = (
|
| 330 |
+
("can_bus", ego_motion_can_bus),
|
| 331 |
+
("img_feat", encoder_image_features),
|
| 332 |
+
("lidar2img", camera_lidar2img_matrices),
|
| 333 |
+
)
|
| 334 |
+
for tensor_name, tensor_value in scene_start_inputs:
|
| 335 |
+
timing["encoder_set_input_ms"] += self._set_input(selected_encoder, tensor_name, tensor_value)
|
| 336 |
+
else:
|
| 337 |
+
if previous_frame_bev_embed is None:
|
| 338 |
+
raise RuntimeError(f"frame{frame_index:03d}: previous BEV state is missing for temporal frame")
|
| 339 |
+
|
| 340 |
+
ego_motion_shift = load_record(asset_records["shift"], repo_root).astype(np.float32)
|
| 341 |
+
ego_rotation_can_bus = load_record(asset_records["rotation_can_bus"], repo_root).astype(np.float32)
|
| 342 |
+
rotate_start = time.perf_counter_ns()
|
| 343 |
+
rotated_previous_bev_embed = rotate_prev_bev(previous_frame_bev_embed, ego_rotation_can_bus)
|
| 344 |
+
timing["prev_bev_rotate_ms"] = elapsed_ms(rotate_start)
|
| 345 |
+
|
| 346 |
+
selected_encoder_name = "encoder_temporal"
|
| 347 |
+
selected_encoder_route = "temporal_reuse_previous_bev"
|
| 348 |
+
selected_encoder = self.interpreters[selected_encoder_name]
|
| 349 |
+
timing["encoder_set_input_ms"] = 0.0
|
| 350 |
+
temporal_inputs = (
|
| 351 |
+
("can_bus", ego_motion_can_bus),
|
| 352 |
+
("img_feat", encoder_image_features),
|
| 353 |
+
("lidar2img", camera_lidar2img_matrices),
|
| 354 |
+
("shift", ego_motion_shift),
|
| 355 |
+
("prev_bev", rotated_previous_bev_embed),
|
| 356 |
+
)
|
| 357 |
+
for tensor_name, tensor_value in temporal_inputs:
|
| 358 |
+
timing["encoder_set_input_ms"] += self._set_input(selected_encoder, tensor_name, tensor_value)
|
| 359 |
+
|
| 360 |
+
timing["encoder_invoke_ms"] = self._invoke(selected_encoder, selected_encoder_name)
|
| 361 |
+
current_frame_bev_embed, timing["encoder_get_output_ms"] = self._get_output(
|
| 362 |
+
selected_encoder,
|
| 363 |
+
"bev_embed",
|
| 364 |
+
SHAPES["bev"],
|
| 365 |
+
)
|
| 366 |
+
|
| 367 |
+
decoder = self.interpreters["decoder"]
|
| 368 |
+
timing["decoder_set_input_ms"] = self._set_input(decoder, "bev_embed", current_frame_bev_embed)
|
| 369 |
+
timing["decoder_invoke_ms"] = self._invoke(decoder, "decoder")
|
| 370 |
+
decoder_cls_scores, timing["decoder_get_cls_ms"] = self._get_output(decoder, "cls_scores", SHAPES["decoder"])
|
| 371 |
+
decoder_bbox_preds, timing["decoder_get_bbox_ms"] = self._get_output(decoder, "bbox_preds", SHAPES["decoder"])
|
| 372 |
+
|
| 373 |
+
timing["qnn_invoke_ms"] = (
|
| 374 |
+
timing["backbone_invoke_ms"]
|
| 375 |
+
+ timing["encoder_invoke_ms"]
|
| 376 |
+
+ timing["decoder_invoke_ms"]
|
| 377 |
+
)
|
| 378 |
+
timing["frame_total_ms"] = elapsed_ms(frame_start)
|
| 379 |
+
frame_result = {
|
| 380 |
+
"frame_index": int(frame_index),
|
| 381 |
+
"sample_token": frame_manifest.get("sample_token"),
|
| 382 |
+
"frame_type": "scene_start" if is_scene_start_frame else "temporal",
|
| 383 |
+
"is_scene_start": is_scene_start_frame,
|
| 384 |
+
"encoder": selected_encoder_name,
|
| 385 |
+
"encoder_route": selected_encoder_route,
|
| 386 |
+
"image_preprocess": image_preprocess,
|
| 387 |
+
"timing_ms": timing,
|
| 388 |
+
"status": "PASS",
|
| 389 |
+
}
|
| 390 |
+
return current_frame_bev_embed, decoder_cls_scores, decoder_bbox_preds, frame_result
|
| 391 |
+
|
| 392 |
+
def run_manifest(
|
| 393 |
+
self,
|
| 394 |
+
manifest_path: str | Path,
|
| 395 |
+
repo_root: str | Path,
|
| 396 |
+
output_dir: str | Path,
|
| 397 |
+
nms_contract_path: str | Path,
|
| 398 |
+
frame_start: int = 0,
|
| 399 |
+
frame_count: int | None = None,
|
| 400 |
+
save_all_raw: bool = False,
|
| 401 |
+
visualize: bool = True,
|
| 402 |
+
vis_score_thr: float = 0.0,
|
| 403 |
+
vis_max_boxes: int = 80,
|
| 404 |
+
check_image_sha: bool = False,
|
| 405 |
+
preprocess_workers: int = 6,
|
| 406 |
+
) -> dict[str, Any]:
|
| 407 |
+
run_manifest_start = time.perf_counter_ns()
|
| 408 |
+
manifest_load_start = time.perf_counter_ns()
|
| 409 |
+
manifest = load_json(manifest_path)
|
| 410 |
+
nms_contract = load_json(nms_contract_path)
|
| 411 |
+
manifest_load_ms = elapsed_ms(manifest_load_start)
|
| 412 |
+
total_frames = int(manifest.get("total_frames", len(manifest["frames"])))
|
| 413 |
+
end = total_frames if frame_count is None else min(total_frames, frame_start + frame_count)
|
| 414 |
+
output_path = Path(output_dir)
|
| 415 |
+
output_path.mkdir(parents=True, exist_ok=True)
|
| 416 |
+
|
| 417 |
+
previous_frame_bev_embed: np.ndarray | None = None
|
| 418 |
+
frames: dict[str, Any] = {}
|
| 419 |
+
scene_start_count = 0
|
| 420 |
+
temporal_count = 0
|
| 421 |
+
frame_ms: list[float] = []
|
| 422 |
+
qnn_invoke_ms: list[float] = []
|
| 423 |
+
backbone_invoke_ms: list[float] = []
|
| 424 |
+
encoder_invoke_ms: list[float] = []
|
| 425 |
+
scene_start_encoder_invoke_ms: list[float] = []
|
| 426 |
+
temporal_encoder_invoke_ms: list[float] = []
|
| 427 |
+
decoder_invoke_ms: list[float] = []
|
| 428 |
+
image_preprocess_ms: list[float] = []
|
| 429 |
+
raw_save_ms: list[float] = []
|
| 430 |
+
nms_decode_ms: list[float] = []
|
| 431 |
+
final_coordinates_save_ms: list[float] = []
|
| 432 |
+
postprocess_ms: list[float] = []
|
| 433 |
+
visualization_ms: list[float] = []
|
| 434 |
+
final_outputs: dict[str, Any] | None = None
|
| 435 |
+
final_coordinates: dict[str, Any] = {}
|
| 436 |
+
visualizations: dict[str, Any] = {}
|
| 437 |
+
|
| 438 |
+
for frame_index in range(frame_start, end):
|
| 439 |
+
sample = f"sample_{frame_index:03d}"
|
| 440 |
+
current_frame_bev_embed, decoder_cls_scores, decoder_bbox_preds, frame_record = self.run_frame(
|
| 441 |
+
frame_index,
|
| 442 |
+
manifest["frames"][sample],
|
| 443 |
+
repo_root,
|
| 444 |
+
previous_frame_bev_embed,
|
| 445 |
+
check_image_sha=check_image_sha,
|
| 446 |
+
preprocess_workers=preprocess_workers,
|
| 447 |
+
)
|
| 448 |
+
previous_frame_bev_embed = np.ascontiguousarray(current_frame_bev_embed, dtype=np.float32)
|
| 449 |
+
frames[sample] = frame_record
|
| 450 |
+
frame_ms.append(frame_record["timing_ms"]["frame_total_ms"])
|
| 451 |
+
qnn_invoke_ms.append(frame_record["timing_ms"]["qnn_invoke_ms"])
|
| 452 |
+
backbone_invoke_ms.append(frame_record["timing_ms"]["backbone_invoke_ms"])
|
| 453 |
+
encoder_time = frame_record["timing_ms"]["encoder_invoke_ms"]
|
| 454 |
+
encoder_invoke_ms.append(encoder_time)
|
| 455 |
+
if frame_record["encoder"] == "encoder_scene_start":
|
| 456 |
+
scene_start_encoder_invoke_ms.append(encoder_time)
|
| 457 |
+
elif frame_record["encoder"] == "encoder_temporal":
|
| 458 |
+
temporal_encoder_invoke_ms.append(encoder_time)
|
| 459 |
+
decoder_invoke_ms.append(frame_record["timing_ms"]["decoder_invoke_ms"])
|
| 460 |
+
image_preprocess_ms.append(frame_record["timing_ms"].get("image_preprocess_ms", 0.0))
|
| 461 |
+
if frame_record["is_scene_start"]:
|
| 462 |
+
scene_start_count += 1
|
| 463 |
+
else:
|
| 464 |
+
temporal_count += 1
|
| 465 |
+
raw_save_value = 0.0
|
| 466 |
+
if save_all_raw or frame_index == end - 1:
|
| 467 |
+
raw_save_start = time.perf_counter_ns()
|
| 468 |
+
final_outputs = save_raw_outputs(output_path, frame_index, decoder_cls_scores, decoder_bbox_preds)
|
| 469 |
+
raw_save_value = elapsed_ms(raw_save_start)
|
| 470 |
+
raw_save_ms.append(raw_save_value)
|
| 471 |
+
|
| 472 |
+
nms_start = time.perf_counter_ns()
|
| 473 |
+
boxes, scores, labels = portable_nms.decode_numpy_nmsfreecoder(
|
| 474 |
+
decoder_cls_scores,
|
| 475 |
+
decoder_bbox_preds,
|
| 476 |
+
nms_contract,
|
| 477 |
+
)
|
| 478 |
+
nms_value = elapsed_ms(nms_start)
|
| 479 |
+
nms_decode_ms.append(nms_value)
|
| 480 |
+
|
| 481 |
+
final_save_start = time.perf_counter_ns()
|
| 482 |
+
final_coordinates[f"frame{frame_index:03d}"] = save_final_coordinates(
|
| 483 |
+
output_path,
|
| 484 |
+
frame_index,
|
| 485 |
+
boxes,
|
| 486 |
+
scores,
|
| 487 |
+
labels,
|
| 488 |
+
)
|
| 489 |
+
final_save_value = elapsed_ms(final_save_start)
|
| 490 |
+
final_coordinates_save_ms.append(final_save_value)
|
| 491 |
+
postprocess_ms.append(raw_save_value + nms_value + final_save_value)
|
| 492 |
+
|
| 493 |
+
visualization_value = 0.0
|
| 494 |
+
if visualize:
|
| 495 |
+
visualization_start = time.perf_counter_ns()
|
| 496 |
+
visualizations[f"frame{frame_index:03d}"] = save_camera_grid_visualization(
|
| 497 |
+
output_path,
|
| 498 |
+
frame_index,
|
| 499 |
+
manifest["frames"][sample],
|
| 500 |
+
repo_root,
|
| 501 |
+
boxes,
|
| 502 |
+
scores,
|
| 503 |
+
labels,
|
| 504 |
+
score_thr=vis_score_thr,
|
| 505 |
+
max_boxes=vis_max_boxes,
|
| 506 |
+
result_dir=output_path,
|
| 507 |
+
)
|
| 508 |
+
visualization_value = elapsed_ms(visualization_start)
|
| 509 |
+
visualization_ms.append(visualization_value)
|
| 510 |
+
frame_record["timing_ms"]["raw_save_ms"] = raw_save_value
|
| 511 |
+
frame_record["timing_ms"]["nms_decode_ms"] = nms_value
|
| 512 |
+
frame_record["timing_ms"]["final_coordinates_save_ms"] = final_save_value
|
| 513 |
+
frame_record["timing_ms"]["postprocess_ms"] = raw_save_value + nms_value + final_save_value
|
| 514 |
+
frame_record["timing_ms"]["visualization_ms"] = visualization_value
|
| 515 |
+
top_detections = []
|
| 516 |
+
for det_index in range(min(5, len(scores))):
|
| 517 |
+
label_id = int(labels[det_index])
|
| 518 |
+
class_name = NUSCENES_CLASSES[label_id] if label_id < len(NUSCENES_CLASSES) else str(label_id)
|
| 519 |
+
top_detections.append({
|
| 520 |
+
"box": [float(value) for value in boxes[det_index].tolist()],
|
| 521 |
+
"score": float(scores[det_index]),
|
| 522 |
+
"label": label_id,
|
| 523 |
+
"class_name": class_name,
|
| 524 |
+
})
|
| 525 |
+
frame_record["detections"] = {
|
| 526 |
+
"count": int(len(scores)),
|
| 527 |
+
"top": top_detections,
|
| 528 |
+
}
|
| 529 |
+
route_text = (
|
| 530 |
+
"first frame: reset prev_bev state"
|
| 531 |
+
if frame_record["encoder_route"] == "scene_start_reset_prev_bev"
|
| 532 |
+
else "later frame: previous bev_embed -> prev_bev"
|
| 533 |
+
)
|
| 534 |
+
print(f"Frame {frame_index:03d} | QNN={frame_record['timing_ms']['qnn_invoke_ms']:.1f} ms backbone={frame_record['timing_ms']['backbone_invoke_ms']:.1f} encoder={frame_record['timing_ms']['encoder_invoke_ms']:.1f} decoder={frame_record['timing_ms']['decoder_invoke_ms']:.1f}")
|
| 535 |
+
|
| 536 |
+
camera_grid_gif = None
|
| 537 |
+
camera_grid_summary = None
|
| 538 |
+
gif_ms = 0.0
|
| 539 |
+
if visualize and visualizations:
|
| 540 |
+
gif_start = time.perf_counter_ns()
|
| 541 |
+
ordered_records = [visualizations[key] for key in sorted(visualizations)]
|
| 542 |
+
camera_grid_gif = save_camera_grid_gif(ordered_records, output_path)
|
| 543 |
+
camera_grid_summary = save_camera_grid_summary(output_path, ordered_records, camera_grid_gif)
|
| 544 |
+
gif_ms = elapsed_ms(gif_start)
|
| 545 |
+
if camera_grid_gif:
|
| 546 |
+
print(f"GIF: {camera_grid_gif['path']}")
|
| 547 |
+
|
| 548 |
+
total_with_visualization_ms = elapsed_ms(run_manifest_start)
|
| 549 |
+
total_without_visualization_ms = (
|
| 550 |
+
manifest_load_ms
|
| 551 |
+
+ sum(frame_ms)
|
| 552 |
+
+ sum(postprocess_ms)
|
| 553 |
+
)
|
| 554 |
+
total_visualization_ms = sum(visualization_ms) + gif_ms
|
| 555 |
+
|
| 556 |
+
return {
|
| 557 |
+
"status": "PASS",
|
| 558 |
+
"manifest": str(Path(manifest_path).resolve()),
|
| 559 |
+
"nms_contract": str(Path(nms_contract_path).resolve()),
|
| 560 |
+
"repo_root": str(Path(repo_root).resolve()),
|
| 561 |
+
"frame_range": [int(frame_start), int(end - 1)] if end > frame_start else [],
|
| 562 |
+
"completed_frames": int(end - frame_start),
|
| 563 |
+
"scene_start_encoder_count": scene_start_count,
|
| 564 |
+
"temporal_encoder_count": temporal_count,
|
| 565 |
+
"model_platform": self.model_platform,
|
| 566 |
+
"models": self.model_records,
|
| 567 |
+
"model_load_timing_ms": self.model_load_timing_ms,
|
| 568 |
+
"end_to_end_timing_ms": {
|
| 569 |
+
"manifest_and_contract_load_ms": manifest_load_ms,
|
| 570 |
+
"complete_inference_no_visualization_ms": total_without_visualization_ms,
|
| 571 |
+
"visualization_total_ms": total_visualization_ms,
|
| 572 |
+
"complete_inference_with_visualization_ms": total_with_visualization_ms,
|
| 573 |
+
"camera_grid_gif_ms": gif_ms,
|
| 574 |
+
},
|
| 575 |
+
"timing_ms": stats(frame_ms),
|
| 576 |
+
"qnn_invoke_ms": stats(qnn_invoke_ms),
|
| 577 |
+
"component_invoke_ms": {
|
| 578 |
+
"backbone": stats(backbone_invoke_ms),
|
| 579 |
+
"encoder": stats(encoder_invoke_ms),
|
| 580 |
+
"encoder_scene_start": stats(scene_start_encoder_invoke_ms),
|
| 581 |
+
"encoder_temporal": stats(temporal_encoder_invoke_ms),
|
| 582 |
+
"decoder": stats(decoder_invoke_ms),
|
| 583 |
+
},
|
| 584 |
+
"per_bin_qnn_invoke_ms": {
|
| 585 |
+
"backbone_context.bin": stats(backbone_invoke_ms),
|
| 586 |
+
"scene_start_encoder_context.bin": stats(scene_start_encoder_invoke_ms),
|
| 587 |
+
"temporal_encoder_context.bin": stats(temporal_encoder_invoke_ms),
|
| 588 |
+
"decoder_context.bin": stats(decoder_invoke_ms),
|
| 589 |
+
},
|
| 590 |
+
"image_preprocess_ms": stats(image_preprocess_ms),
|
| 591 |
+
"postprocess_ms": stats(postprocess_ms),
|
| 592 |
+
"nms_decode_ms": stats(nms_decode_ms),
|
| 593 |
+
"raw_save_ms": stats(raw_save_ms),
|
| 594 |
+
"final_coordinates_save_ms": stats(final_coordinates_save_ms),
|
| 595 |
+
"visualization_ms": stats(visualization_ms),
|
| 596 |
+
"timing_contract": {
|
| 597 |
+
"frame_total_ms": "Per-frame inference path: image preprocessing plus model input/output and QNN invoke time; excludes NMS, result saving, and camera-grid visualization rendering.",
|
| 598 |
+
"qnn_invoke_ms": "Backbone + selected encoder + decoder invoke time only. Excludes preprocessing, tensor set/get, NMS, result saving, and visualization.",
|
| 599 |
+
"per_bin_qnn_invoke_ms": "Pure AidLite/QNN interpreter.invoke() time grouped by delivered .bin context file.",
|
| 600 |
+
"image_preprocess_ms": "Six-camera JPG decode, RGB conversion, normalization, resize, CHW conversion, and zero padding.",
|
| 601 |
+
"complete_inference_no_visualization_ms": "Manifest load plus all frame inference, NMS, and result saving; excludes PNG/GIF rendering.",
|
| 602 |
+
"complete_inference_with_visualization_ms": "Full run_manifest wall time including PNG/GIF rendering; excludes Python process startup and model loading, which are reported separately.",
|
| 603 |
+
},
|
| 604 |
+
"frames": frames,
|
| 605 |
+
"final_outputs": final_outputs,
|
| 606 |
+
"final_coordinates": final_coordinates,
|
| 607 |
+
"visualizations": visualizations,
|
| 608 |
+
"camera_grid_gif": camera_grid_gif,
|
| 609 |
+
"camera_grid_summary": camera_grid_summary,
|
| 610 |
+
}
|
| 611 |
+
|
targets/QCS8550/FP16/code/python/camera_grid_visualization.py
ADDED
|
@@ -0,0 +1,309 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
from __future__ import annotations
|
| 2 |
+
|
| 3 |
+
import json
|
| 4 |
+
import math
|
| 5 |
+
from pathlib import Path
|
| 6 |
+
from typing import Any, Sequence
|
| 7 |
+
|
| 8 |
+
import numpy as np
|
| 9 |
+
from PIL import Image, ImageDraw, ImageFont
|
| 10 |
+
|
| 11 |
+
from utils import load_record, resolve_path, sha256_file
|
| 12 |
+
|
| 13 |
+
|
| 14 |
+
CLASS_NAMES = (
|
| 15 |
+
"car",
|
| 16 |
+
"truck",
|
| 17 |
+
"construction_vehicle",
|
| 18 |
+
"bus",
|
| 19 |
+
"trailer",
|
| 20 |
+
"barrier",
|
| 21 |
+
"motorcycle",
|
| 22 |
+
"bicycle",
|
| 23 |
+
"pedestrian",
|
| 24 |
+
"traffic_cone",
|
| 25 |
+
)
|
| 26 |
+
|
| 27 |
+
MODEL_CAMERA_ORDER = (
|
| 28 |
+
"CAM_FRONT",
|
| 29 |
+
"CAM_FRONT_RIGHT",
|
| 30 |
+
"CAM_FRONT_LEFT",
|
| 31 |
+
"CAM_BACK",
|
| 32 |
+
"CAM_BACK_LEFT",
|
| 33 |
+
"CAM_BACK_RIGHT",
|
| 34 |
+
)
|
| 35 |
+
|
| 36 |
+
GRID_CAMERA_ORDER = (
|
| 37 |
+
"CAM_FRONT_LEFT",
|
| 38 |
+
"CAM_FRONT",
|
| 39 |
+
"CAM_FRONT_RIGHT",
|
| 40 |
+
"CAM_BACK_LEFT",
|
| 41 |
+
"CAM_BACK",
|
| 42 |
+
"CAM_BACK_RIGHT",
|
| 43 |
+
)
|
| 44 |
+
|
| 45 |
+
EDGES = (
|
| 46 |
+
(0, 1), (1, 2), (2, 3), (3, 0),
|
| 47 |
+
(4, 5), (5, 6), (6, 7), (7, 4),
|
| 48 |
+
(0, 4), (1, 5), (2, 6), (3, 7),
|
| 49 |
+
)
|
| 50 |
+
|
| 51 |
+
|
| 52 |
+
|
| 53 |
+
def load_font(size: int = 18):
|
| 54 |
+
for path in (
|
| 55 |
+
"/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf",
|
| 56 |
+
"/usr/share/fonts/truetype/liberation2/LiberationSans-Regular.ttf",
|
| 57 |
+
):
|
| 58 |
+
if Path(path).is_file():
|
| 59 |
+
return ImageFont.truetype(path, size)
|
| 60 |
+
return ImageFont.load_default()
|
| 61 |
+
|
| 62 |
+
|
| 63 |
+
def corners_3d(box: np.ndarray) -> np.ndarray:
|
| 64 |
+
x, y, z, w, length, h, yaw = [float(v) for v in box[:7]]
|
| 65 |
+
x_c = np.array([length / 2, length / 2, -length / 2, -length / 2, length / 2, length / 2, -length / 2, -length / 2])
|
| 66 |
+
y_c = np.array([w / 2, -w / 2, -w / 2, w / 2, w / 2, -w / 2, -w / 2, w / 2])
|
| 67 |
+
z_c = np.array([h / 2, h / 2, h / 2, h / 2, -h / 2, -h / 2, -h / 2, -h / 2])
|
| 68 |
+
rot = np.array(
|
| 69 |
+
[
|
| 70 |
+
[math.cos(yaw), -math.sin(yaw), 0.0],
|
| 71 |
+
[math.sin(yaw), math.cos(yaw), 0.0],
|
| 72 |
+
[0.0, 0.0, 1.0],
|
| 73 |
+
],
|
| 74 |
+
dtype=np.float64,
|
| 75 |
+
)
|
| 76 |
+
return (rot @ np.stack([x_c, y_c, z_c], axis=0)).T + np.array([x, y, z], dtype=np.float64)
|
| 77 |
+
|
| 78 |
+
|
| 79 |
+
def project_box(box: np.ndarray, lidar2img: np.ndarray, image_size: tuple[int, int]) -> np.ndarray | None:
|
| 80 |
+
corners = corners_3d(box)
|
| 81 |
+
homo = np.concatenate([corners, np.ones((8, 1), dtype=np.float64)], axis=1)
|
| 82 |
+
proj = (lidar2img.astype(np.float64) @ homo.T).T
|
| 83 |
+
depth = proj[:, 2]
|
| 84 |
+
if np.count_nonzero(depth > 1e-3) < 8:
|
| 85 |
+
return None
|
| 86 |
+
points = proj[:, :2] / depth[:, None]
|
| 87 |
+
width, height = image_size
|
| 88 |
+
if not (
|
| 89 |
+
(points[:, 0] >= -100).any()
|
| 90 |
+
and (points[:, 0] <= width + 100).any()
|
| 91 |
+
and (points[:, 1] >= -100).any()
|
| 92 |
+
and (points[:, 1] <= height + 100).any()
|
| 93 |
+
):
|
| 94 |
+
return None
|
| 95 |
+
return points
|
| 96 |
+
|
| 97 |
+
|
| 98 |
+
def draw_projected_box(draw: ImageDraw.ImageDraw, points: np.ndarray, color: tuple[int, int, int], width: int = 3) -> None:
|
| 99 |
+
xy = [(float(x), float(y)) for x, y in points]
|
| 100 |
+
for i, j in EDGES:
|
| 101 |
+
draw.line([xy[i], xy[j]], fill=color, width=width)
|
| 102 |
+
|
| 103 |
+
|
| 104 |
+
def draw_label(draw: ImageDraw.ImageDraw, xy: tuple[float, float], text: str, color: tuple[int, int, int], font) -> None:
|
| 105 |
+
x = max(0.0, min(float(xy[0]), 790.0))
|
| 106 |
+
y = max(0.0, min(float(xy[1]), 440.0))
|
| 107 |
+
bbox = draw.textbbox((x, y), text, font=font)
|
| 108 |
+
draw.rectangle(bbox, fill=(0, 0, 0))
|
| 109 |
+
draw.text((x, y), text, fill=color, font=font)
|
| 110 |
+
|
| 111 |
+
|
| 112 |
+
def select_predictions(
|
| 113 |
+
boxes: np.ndarray,
|
| 114 |
+
scores: np.ndarray,
|
| 115 |
+
labels: np.ndarray,
|
| 116 |
+
score_thr: float,
|
| 117 |
+
max_boxes: int,
|
| 118 |
+
) -> list[tuple[np.ndarray, float, str]]:
|
| 119 |
+
boxes = np.asarray(boxes, dtype=np.float64)
|
| 120 |
+
scores = np.asarray(scores, dtype=np.float64).reshape(-1)
|
| 121 |
+
labels = np.asarray(labels, dtype=np.int64).reshape(-1)
|
| 122 |
+
order = np.argsort(-scores, kind="stable")
|
| 123 |
+
out: list[tuple[np.ndarray, float, str]] = []
|
| 124 |
+
for idx in order:
|
| 125 |
+
if float(scores[idx]) < float(score_thr):
|
| 126 |
+
continue
|
| 127 |
+
label = int(labels[idx])
|
| 128 |
+
name = CLASS_NAMES[label] if 0 <= label < len(CLASS_NAMES) else str(label)
|
| 129 |
+
out.append((boxes[idx], float(scores[idx]), name))
|
| 130 |
+
if len(out) >= int(max_boxes):
|
| 131 |
+
break
|
| 132 |
+
return out
|
| 133 |
+
|
| 134 |
+
|
| 135 |
+
def camera_records(frame_assets: dict[str, Any], repo_root: str | Path) -> dict[str, Path]:
|
| 136 |
+
record = frame_assets["assets"]["camera_images"]
|
| 137 |
+
return {item["name"]: resolve_path(item["path"], repo_root) for item in record["images"]}
|
| 138 |
+
|
| 139 |
+
|
| 140 |
+
def load_lidar2img_by_camera(frame_assets: dict[str, Any], repo_root: str | Path) -> dict[str, np.ndarray]:
|
| 141 |
+
value = load_record(frame_assets["assets"]["lidar2img"], repo_root).astype(np.float64).reshape(1, 6, 4, 4)[0]
|
| 142 |
+
return {name: value[index] for index, name in enumerate(MODEL_CAMERA_ORDER)}
|
| 143 |
+
|
| 144 |
+
|
| 145 |
+
def wrap_text(text: str, width: int = 42) -> list[str]:
|
| 146 |
+
words = str(text).split()
|
| 147 |
+
lines: list[str] = []
|
| 148 |
+
current = ""
|
| 149 |
+
for word in words:
|
| 150 |
+
trial = word if not current else current + " " + word
|
| 151 |
+
if len(trial) <= width:
|
| 152 |
+
current = trial
|
| 153 |
+
else:
|
| 154 |
+
if current:
|
| 155 |
+
lines.append(current)
|
| 156 |
+
current = word
|
| 157 |
+
if current:
|
| 158 |
+
lines.append(current)
|
| 159 |
+
return lines
|
| 160 |
+
|
| 161 |
+
|
| 162 |
+
def add_sidebar(sheet: Image.Image, frame_index: int, branch: str, score_thr: float, max_boxes: int, result_dir: str | Path | None) -> Image.Image:
|
| 163 |
+
sidebar_w = 520
|
| 164 |
+
canvas = Image.new("RGB", (sheet.width + sidebar_w, sheet.height), (250, 250, 250))
|
| 165 |
+
canvas.paste(sheet, (0, 0))
|
| 166 |
+
draw = ImageDraw.Draw(canvas)
|
| 167 |
+
title_font = load_font(24)
|
| 168 |
+
body_font = load_font(18)
|
| 169 |
+
small_font = load_font(15)
|
| 170 |
+
x0 = sheet.width + 24
|
| 171 |
+
y = 24
|
| 172 |
+
draw.rectangle((sheet.width, 0, canvas.width - 1, canvas.height - 1), fill=(245, 247, 250), outline=(180, 180, 180), width=2)
|
| 173 |
+
draw.text((x0, y), "Final Detection Visualization", fill=(0, 0, 0), font=title_font)
|
| 174 |
+
y += 46
|
| 175 |
+
lines = [
|
| 176 |
+
f"frame{frame_index:03d} / {branch}",
|
| 177 |
+
"Final Model: W8A8 per_channel",
|
| 178 |
+
"Role: AidLite/QNN board demo",
|
| 179 |
+
"Input: six raw camera JPGs",
|
| 180 |
+
"Preprocess: BGR->RGB normalize resize CHW pad",
|
| 181 |
+
"Temporal: scene-start reset, then prev_bev recursion",
|
| 182 |
+
f"Score threshold: {score_thr}",
|
| 183 |
+
f"Max boxes: {max_boxes}",
|
| 184 |
+
"Red: W8A8 final detections",
|
| 185 |
+
]
|
| 186 |
+
for raw in lines:
|
| 187 |
+
color = (0, 0, 0)
|
| 188 |
+
if raw.startswith("Final"):
|
| 189 |
+
color = (180, 0, 0)
|
| 190 |
+
elif raw.startswith("Red"):
|
| 191 |
+
color = (220, 20, 20)
|
| 192 |
+
for line in wrap_text(raw, width=42):
|
| 193 |
+
draw.text((x0, y), line, fill=color, font=body_font)
|
| 194 |
+
y += 27
|
| 195 |
+
y += 4
|
| 196 |
+
if result_dir is not None:
|
| 197 |
+
y += 14
|
| 198 |
+
draw.text((x0, y), "Input result directory:", fill=(0, 0, 0), font=body_font)
|
| 199 |
+
y += 28
|
| 200 |
+
for line in wrap_text(str(result_dir), width=48):
|
| 201 |
+
draw.text((x0, y), line, fill=(80, 80, 80), font=small_font)
|
| 202 |
+
y += 22
|
| 203 |
+
return canvas
|
| 204 |
+
|
| 205 |
+
|
| 206 |
+
def save_camera_grid_visualization(
|
| 207 |
+
output_dir: str | Path,
|
| 208 |
+
frame_index: int,
|
| 209 |
+
frame_assets: dict[str, Any],
|
| 210 |
+
repo_root: str | Path,
|
| 211 |
+
boxes: np.ndarray,
|
| 212 |
+
scores: np.ndarray,
|
| 213 |
+
labels: np.ndarray,
|
| 214 |
+
score_thr: float = 0.25,
|
| 215 |
+
max_boxes: int = 80,
|
| 216 |
+
panel_width: int = 900,
|
| 217 |
+
draw_labels: bool = False,
|
| 218 |
+
metrics_panel: bool = True,
|
| 219 |
+
result_dir: str | Path | None = None,
|
| 220 |
+
) -> dict[str, Any]:
|
| 221 |
+
output_path = Path(output_dir)
|
| 222 |
+
output_path.mkdir(parents=True, exist_ok=True)
|
| 223 |
+
font = load_font(18)
|
| 224 |
+
label_font = load_font(15)
|
| 225 |
+
cameras = camera_records(frame_assets, repo_root)
|
| 226 |
+
lidar2img = load_lidar2img_by_camera(frame_assets, repo_root)
|
| 227 |
+
preds = select_predictions(boxes, scores, labels, score_thr, max_boxes)
|
| 228 |
+
|
| 229 |
+
panels = []
|
| 230 |
+
per_camera = {}
|
| 231 |
+
for camera_name in GRID_CAMERA_ORDER:
|
| 232 |
+
image = Image.open(cameras[camera_name]).convert("RGB").resize((800, 450))
|
| 233 |
+
draw = ImageDraw.Draw(image)
|
| 234 |
+
count = 0
|
| 235 |
+
for box, score, class_name in preds:
|
| 236 |
+
points = project_box(box, lidar2img[camera_name], image.size)
|
| 237 |
+
if points is None:
|
| 238 |
+
continue
|
| 239 |
+
draw_projected_box(draw, points, (255, 40, 40), width=3)
|
| 240 |
+
if draw_labels:
|
| 241 |
+
draw_label(draw, tuple(points[:, :2].min(axis=0)), f"W8A8 {class_name} {score:.2f}", (255, 180, 180), label_font)
|
| 242 |
+
count += 1
|
| 243 |
+
header = Image.new("RGB", (image.width, 44), (20, 20, 20))
|
| 244 |
+
hdraw = ImageDraw.Draw(header)
|
| 245 |
+
hdraw.text((10, 10), f"{camera_name} | W8A8 {count}", fill=(255, 255, 255), font=font)
|
| 246 |
+
panel = Image.new("RGB", (image.width, image.height + header.height), (0, 0, 0))
|
| 247 |
+
panel.paste(header, (0, 0))
|
| 248 |
+
panel.paste(image, (0, header.height))
|
| 249 |
+
panel = panel.resize((panel_width, int(panel.height * panel_width / panel.width)))
|
| 250 |
+
panels.append(panel)
|
| 251 |
+
per_camera[camera_name] = {"w8a8": int(count)}
|
| 252 |
+
|
| 253 |
+
cols, rows = 3, 2
|
| 254 |
+
panel_w, panel_h = panels[0].size
|
| 255 |
+
title_h = 70
|
| 256 |
+
sheet = Image.new("RGB", (cols * panel_w, rows * panel_h + title_h), (245, 245, 245))
|
| 257 |
+
sheet_draw = ImageDraw.Draw(sheet)
|
| 258 |
+
branch = "scene_start" if bool(frame_assets.get("is_scene_start", False)) else "temporal"
|
| 259 |
+
token = str(frame_assets.get("sample_token", ""))[:16]
|
| 260 |
+
title = f"Sample4 frame{frame_index:03d} ({branch}) token={token} | red=W8A8 final | score_thr={score_thr}"
|
| 261 |
+
sheet_draw.text((16, 18), title, fill=(0, 0, 0), font=font)
|
| 262 |
+
for index, panel in enumerate(panels):
|
| 263 |
+
x = (index % cols) * panel_w
|
| 264 |
+
y = title_h + (index // cols) * panel_h
|
| 265 |
+
sheet.paste(panel, (x, y))
|
| 266 |
+
|
| 267 |
+
if metrics_panel:
|
| 268 |
+
sheet = add_sidebar(sheet, frame_index, branch, score_thr, max_boxes, result_dir)
|
| 269 |
+
|
| 270 |
+
out_file = output_path / f"frame{frame_index:03d}_camera_grid.png"
|
| 271 |
+
sheet.save(out_file)
|
| 272 |
+
return {
|
| 273 |
+
"path": str(out_file),
|
| 274 |
+
"sha256": sha256_file(out_file),
|
| 275 |
+
"mode": "camera_grid_w8a8_only",
|
| 276 |
+
"score_thr": float(score_thr),
|
| 277 |
+
"max_boxes": int(max_boxes),
|
| 278 |
+
"candidate_count": int(len(preds)),
|
| 279 |
+
"per_camera_projected_counts": per_camera,
|
| 280 |
+
}
|
| 281 |
+
|
| 282 |
+
|
| 283 |
+
def save_camera_grid_gif(records: Sequence[dict[str, Any]], output_dir: str | Path, name: str = "sample4_camera_grid.gif", width: int = 1800, duration_ms: int = 700) -> dict[str, Any] | None:
|
| 284 |
+
paths = [Path(record["path"]) for record in records if record.get("path")]
|
| 285 |
+
if not paths:
|
| 286 |
+
return None
|
| 287 |
+
frames = []
|
| 288 |
+
for path in paths:
|
| 289 |
+
image = Image.open(path).convert("RGB")
|
| 290 |
+
height = int(image.height * width / image.width)
|
| 291 |
+
frames.append(image.resize((width, height)))
|
| 292 |
+
out_file = Path(output_dir) / name
|
| 293 |
+
frames[0].save(out_file, save_all=True, append_images=frames[1:], duration=duration_ms, loop=0)
|
| 294 |
+
return {"path": str(out_file), "sha256": sha256_file(out_file), "frame_count": len(frames)}
|
| 295 |
+
|
| 296 |
+
|
| 297 |
+
def save_camera_grid_summary(output_dir: str | Path, records: Sequence[dict[str, Any]], gif_record: dict[str, Any] | None) -> dict[str, Any]:
|
| 298 |
+
output_path = Path(output_dir)
|
| 299 |
+
summary = {
|
| 300 |
+
"status": "PASS",
|
| 301 |
+
"mode": "sample4_camera_grid_w8a8_only",
|
| 302 |
+
"visualizations": list(records),
|
| 303 |
+
"gif": gif_record,
|
| 304 |
+
}
|
| 305 |
+
path = output_path / "sample4_camera_grid_summary.json"
|
| 306 |
+
path.write_text(json.dumps(summary, indent=2, ensure_ascii=False) + "\n", encoding="utf-8")
|
| 307 |
+
summary["path"] = str(path)
|
| 308 |
+
summary["sha256"] = sha256_file(path)
|
| 309 |
+
return summary
|
targets/QCS8550/FP16/code/python/configs/demo_config.json
ADDED
|
@@ -0,0 +1,34 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"_comment": "BEVFormer board demo runtime config. Paths are relative to the bevformer_delivery_demo root.",
|
| 3 |
+
"name": "bevformer_delivery_demo",
|
| 4 |
+
"default_variant": "strict_w8a8_train64",
|
| 5 |
+
"model_type": "QNN248",
|
| 6 |
+
"models": {
|
| 7 |
+
"_comment": "Model paths are resolved automatically from models/<platform>/FP16, or from the four CLI model path arguments. Supported platforms: QCS8550, IQ9, QCS8625, IQ8."
|
| 8 |
+
},
|
| 9 |
+
"inputs": {
|
| 10 |
+
"_comment": "The bundled sample is four continuous frames from raw six-camera JPGs.",
|
| 11 |
+
"asset_manifest": "code/python/datasets/sample4/asset_manifest.json",
|
| 12 |
+
"frame_count": 4,
|
| 13 |
+
"scene_start_indices": [
|
| 14 |
+
0
|
| 15 |
+
],
|
| 16 |
+
"preprocess_workers": 6
|
| 17 |
+
},
|
| 18 |
+
"postprocess": {
|
| 19 |
+
"_comment": "NMSFreeCoder contract and saved detection output format.",
|
| 20 |
+
"nms_contract": "code/python/configs/nms_runtime_contract.json",
|
| 21 |
+
"output": "frameXXX_final_coordinates.npz"
|
| 22 |
+
},
|
| 23 |
+
"visualization": {
|
| 24 |
+
"_comment": "Camera-grid visualization generated from W8A8 detections and bundled camera JPGs.",
|
| 25 |
+
"enabled_by_default": true,
|
| 26 |
+
"output": "frameXXX_camera_grid.png + sample4_camera_grid.gif",
|
| 27 |
+
"score_thr": 0.25,
|
| 28 |
+
"max_boxes": 80
|
| 29 |
+
},
|
| 30 |
+
"outputs": {
|
| 31 |
+
"_comment": "Default output root. If --output_dir is omitted, each run is saved under outputs/YYYY_MM_DD_HH_MM.",
|
| 32 |
+
"default_dir": "outputs"
|
| 33 |
+
}
|
| 34 |
+
}
|
targets/QCS8550/FP16/code/python/configs/nms_runtime_contract.json
ADDED
|
@@ -0,0 +1,11 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"type": "NMSFreeCoder",
|
| 3 |
+
"num_classes": 10,
|
| 4 |
+
"max_num": 300,
|
| 5 |
+
"num_query": 900,
|
| 6 |
+
"code_size": 10,
|
| 7 |
+
"pc_range": [-51.2, -51.2, -5.0, 51.2, 51.2, 3.0],
|
| 8 |
+
"post_center_range": [-61.2, -61.2, -10.0, 61.2, 61.2, 10.0],
|
| 9 |
+
"score_threshold": null,
|
| 10 |
+
"selected_sigmoid_mode": "direct"
|
| 11 |
+
}
|
targets/QCS8550/FP16/code/python/datasets/sample4/README.md
ADDED
|
@@ -0,0 +1,10 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# BEVFormer Sample4 Dataset
|
| 2 |
+
|
| 3 |
+
Self-contained four-frame continuous sample for the AidLite board demo.
|
| 4 |
+
|
| 5 |
+
- Each frame contains six original nuScenes camera JPG files under `cameras/`.
|
| 6 |
+
- `python/utils.py` preprocesses those JPG files on the board before backbone inference.
|
| 7 |
+
- Small raw tensors are still included for BEVFormer metadata: `can_bus`, `shift`, `lidar2img`, and temporal rotation inputs.
|
| 8 |
+
- frame 000 uses `encoder_scene_start` semantics.
|
| 9 |
+
- frames 001-003 use `encoder_temporal` with the previous live `bev_embed`.
|
| 10 |
+
- Paths in `asset_manifest.json` are relative to `/home/aidlux` when copied to the board.
|
targets/QCS8550/FP16/code/python/datasets/sample4/asset_manifest.json
ADDED
|
@@ -0,0 +1,535 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"audit_type": "BEVFORMER_DELIVERY_DEMO_SAMPLE4_MANIFEST",
|
| 3 |
+
"status": "PASS",
|
| 4 |
+
"mode": "self_contained_sample4_raw_camera_to_detection",
|
| 5 |
+
"total_frames": 4,
|
| 6 |
+
"scene_start_indices": [
|
| 7 |
+
0
|
| 8 |
+
],
|
| 9 |
+
"frame_range": [
|
| 10 |
+
0,
|
| 11 |
+
3
|
| 12 |
+
],
|
| 13 |
+
"note": "Self-contained 4-frame continuous board demo sample. Image input starts from bundled raw camera JPEGs; Python preprocessing runs on board.",
|
| 14 |
+
"frames": {
|
| 15 |
+
"sample_000": {
|
| 16 |
+
"dataset_index": 0,
|
| 17 |
+
"sample_token": "3e8750f331d7499e9b5123e9eb70f2e2",
|
| 18 |
+
"scene_token": "fcbccedd61424f1b85dcbf8f897f9754",
|
| 19 |
+
"prev_token": "",
|
| 20 |
+
"next_token": "3950bd41f74548429c0f7700ff3d8269",
|
| 21 |
+
"is_scene_start": true,
|
| 22 |
+
"assets": {
|
| 23 |
+
"can_bus": {
|
| 24 |
+
"dtype": "float16",
|
| 25 |
+
"shape": [
|
| 26 |
+
1,
|
| 27 |
+
18
|
| 28 |
+
],
|
| 29 |
+
"bytes": 36,
|
| 30 |
+
"path": "code/python/datasets/sample4/frames/sample_000/can_bus.raw",
|
| 31 |
+
"sha256": "974730a9cf60a4a0cae487ed4b296faf6621abec0bcd7276020ade042fd8dc3b"
|
| 32 |
+
},
|
| 33 |
+
"lidar2img": {
|
| 34 |
+
"dtype": "float16",
|
| 35 |
+
"shape": [
|
| 36 |
+
1,
|
| 37 |
+
6,
|
| 38 |
+
4,
|
| 39 |
+
4
|
| 40 |
+
],
|
| 41 |
+
"bytes": 192,
|
| 42 |
+
"path": "code/python/datasets/sample4/frames/sample_000/lidar2img.raw",
|
| 43 |
+
"sha256": "f1df04f672d707f2fbc8ea9799ee84d8795a1fe22f17ed89c08327e63f663501"
|
| 44 |
+
},
|
| 45 |
+
"camera_images": {
|
| 46 |
+
"type": "raw_camera_jpeg_multiview",
|
| 47 |
+
"order": [
|
| 48 |
+
"CAM_FRONT",
|
| 49 |
+
"CAM_FRONT_RIGHT",
|
| 50 |
+
"CAM_FRONT_LEFT",
|
| 51 |
+
"CAM_BACK",
|
| 52 |
+
"CAM_BACK_LEFT",
|
| 53 |
+
"CAM_BACK_RIGHT"
|
| 54 |
+
],
|
| 55 |
+
"decode": "cv2.imread BGR",
|
| 56 |
+
"pipeline": "BGR -> RGB -> normalize(mean/std) -> resize(800,450) -> CHW -> pad to 480 rows",
|
| 57 |
+
"mean": [
|
| 58 |
+
123.675,
|
| 59 |
+
116.28,
|
| 60 |
+
103.53
|
| 61 |
+
],
|
| 62 |
+
"std": [
|
| 63 |
+
58.395,
|
| 64 |
+
57.12,
|
| 65 |
+
57.375
|
| 66 |
+
],
|
| 67 |
+
"to_rgb": true,
|
| 68 |
+
"resize": [
|
| 69 |
+
800,
|
| 70 |
+
450
|
| 71 |
+
],
|
| 72 |
+
"output_shape_unpadded": [
|
| 73 |
+
6,
|
| 74 |
+
3,
|
| 75 |
+
450,
|
| 76 |
+
800
|
| 77 |
+
],
|
| 78 |
+
"output_shape_padded": [
|
| 79 |
+
6,
|
| 80 |
+
3,
|
| 81 |
+
480,
|
| 82 |
+
800
|
| 83 |
+
],
|
| 84 |
+
"reference_preprocessed_sha256": "87de4dfddc0cb554908b8c9a89d76125e8698bfbf5ece9d366de924a938d4b1d",
|
| 85 |
+
"images": [
|
| 86 |
+
{
|
| 87 |
+
"name": "CAM_FRONT",
|
| 88 |
+
"path": "code/python/datasets/sample4/frames/sample_000/cameras/CAM_FRONT.jpg",
|
| 89 |
+
"sha256": "0e2067a1618c4424050c45c393ada8c9dbf130fff8d6015004fce901dc07984d",
|
| 90 |
+
"bytes": 141976
|
| 91 |
+
},
|
| 92 |
+
{
|
| 93 |
+
"name": "CAM_FRONT_RIGHT",
|
| 94 |
+
"path": "code/python/datasets/sample4/frames/sample_000/cameras/CAM_FRONT_RIGHT.jpg",
|
| 95 |
+
"sha256": "15cf59c79b086d14da6fae5a7ee5b0f4d0fac0c0a22c0a85faaa5fb215521191",
|
| 96 |
+
"bytes": 147910
|
| 97 |
+
},
|
| 98 |
+
{
|
| 99 |
+
"name": "CAM_FRONT_LEFT",
|
| 100 |
+
"path": "code/python/datasets/sample4/frames/sample_000/cameras/CAM_FRONT_LEFT.jpg",
|
| 101 |
+
"sha256": "8c52f73a3ef06a129c27ef51ac65704428caebfaa60c52f14a5953bbb0124a18",
|
| 102 |
+
"bytes": 144177
|
| 103 |
+
},
|
| 104 |
+
{
|
| 105 |
+
"name": "CAM_BACK",
|
| 106 |
+
"path": "code/python/datasets/sample4/frames/sample_000/cameras/CAM_BACK.jpg",
|
| 107 |
+
"sha256": "047dd43f3f9fb7cc426b1d4ae369181242c5555b84baff5d7e1cbd0dc8b07259",
|
| 108 |
+
"bytes": 108796
|
| 109 |
+
},
|
| 110 |
+
{
|
| 111 |
+
"name": "CAM_BACK_LEFT",
|
| 112 |
+
"path": "code/python/datasets/sample4/frames/sample_000/cameras/CAM_BACK_LEFT.jpg",
|
| 113 |
+
"sha256": "27b2826991733662e4f17c509ad5a66e85e0ca66a6413063d8db9645fa608c11",
|
| 114 |
+
"bytes": 110941
|
| 115 |
+
},
|
| 116 |
+
{
|
| 117 |
+
"name": "CAM_BACK_RIGHT",
|
| 118 |
+
"path": "code/python/datasets/sample4/frames/sample_000/cameras/CAM_BACK_RIGHT.jpg",
|
| 119 |
+
"sha256": "a998caf0e327816b82d2a142d2581a8c1f9c4e67d50b4af3718a381337b7e415",
|
| 120 |
+
"bytes": 124547
|
| 121 |
+
}
|
| 122 |
+
]
|
| 123 |
+
}
|
| 124 |
+
}
|
| 125 |
+
},
|
| 126 |
+
"sample_001": {
|
| 127 |
+
"dataset_index": 1,
|
| 128 |
+
"sample_token": "3950bd41f74548429c0f7700ff3d8269",
|
| 129 |
+
"scene_token": "fcbccedd61424f1b85dcbf8f897f9754",
|
| 130 |
+
"prev_token": "3e8750f331d7499e9b5123e9eb70f2e2",
|
| 131 |
+
"next_token": "c5f58c19249d4137ae063b0e9ecd8b8e",
|
| 132 |
+
"is_scene_start": false,
|
| 133 |
+
"assets": {
|
| 134 |
+
"can_bus": {
|
| 135 |
+
"dtype": "float16",
|
| 136 |
+
"shape": [
|
| 137 |
+
1,
|
| 138 |
+
18
|
| 139 |
+
],
|
| 140 |
+
"bytes": 36,
|
| 141 |
+
"path": "code/python/datasets/sample4/frames/sample_001/can_bus.raw",
|
| 142 |
+
"sha256": "8ece2acd67155365b9394e1569b8242d69f3d9805cea19a5153cb51e79e55152"
|
| 143 |
+
},
|
| 144 |
+
"shift": {
|
| 145 |
+
"dtype": "float16",
|
| 146 |
+
"shape": [
|
| 147 |
+
1,
|
| 148 |
+
2
|
| 149 |
+
],
|
| 150 |
+
"bytes": 4,
|
| 151 |
+
"path": "code/python/datasets/sample4/frames/sample_001/shift.raw",
|
| 152 |
+
"sha256": "530492c17f13ce48ec402622565d7a4ba0987a57b988a3c4d824cc4987dfd5fc"
|
| 153 |
+
},
|
| 154 |
+
"lidar2img": {
|
| 155 |
+
"dtype": "float16",
|
| 156 |
+
"shape": [
|
| 157 |
+
1,
|
| 158 |
+
6,
|
| 159 |
+
4,
|
| 160 |
+
4
|
| 161 |
+
],
|
| 162 |
+
"bytes": 192,
|
| 163 |
+
"path": "code/python/datasets/sample4/frames/sample_001/lidar2img.raw",
|
| 164 |
+
"sha256": "63d7bcbf3a9c63e1184947d9e5112fe30bf4e8533ea29df44daeef3cb610a2ed"
|
| 165 |
+
},
|
| 166 |
+
"rotation_can_bus": {
|
| 167 |
+
"dtype": "float32",
|
| 168 |
+
"shape": [
|
| 169 |
+
1,
|
| 170 |
+
18
|
| 171 |
+
],
|
| 172 |
+
"bytes": 72,
|
| 173 |
+
"path": "code/python/datasets/sample4/frames/sample_001/rotation_can_bus.raw",
|
| 174 |
+
"sha256": "741d75aad896829e7246d446635bc21331242c7e28ef869f986d70e048d40962"
|
| 175 |
+
},
|
| 176 |
+
"camera_images": {
|
| 177 |
+
"type": "raw_camera_jpeg_multiview",
|
| 178 |
+
"order": [
|
| 179 |
+
"CAM_FRONT",
|
| 180 |
+
"CAM_FRONT_RIGHT",
|
| 181 |
+
"CAM_FRONT_LEFT",
|
| 182 |
+
"CAM_BACK",
|
| 183 |
+
"CAM_BACK_LEFT",
|
| 184 |
+
"CAM_BACK_RIGHT"
|
| 185 |
+
],
|
| 186 |
+
"decode": "cv2.imread BGR",
|
| 187 |
+
"pipeline": "BGR -> RGB -> normalize(mean/std) -> resize(800,450) -> CHW -> pad to 480 rows",
|
| 188 |
+
"mean": [
|
| 189 |
+
123.675,
|
| 190 |
+
116.28,
|
| 191 |
+
103.53
|
| 192 |
+
],
|
| 193 |
+
"std": [
|
| 194 |
+
58.395,
|
| 195 |
+
57.12,
|
| 196 |
+
57.375
|
| 197 |
+
],
|
| 198 |
+
"to_rgb": true,
|
| 199 |
+
"resize": [
|
| 200 |
+
800,
|
| 201 |
+
450
|
| 202 |
+
],
|
| 203 |
+
"output_shape_unpadded": [
|
| 204 |
+
6,
|
| 205 |
+
3,
|
| 206 |
+
450,
|
| 207 |
+
800
|
| 208 |
+
],
|
| 209 |
+
"output_shape_padded": [
|
| 210 |
+
6,
|
| 211 |
+
3,
|
| 212 |
+
480,
|
| 213 |
+
800
|
| 214 |
+
],
|
| 215 |
+
"reference_preprocessed_sha256": "ecc0a38ce961abecd5063c3b5c8cb543a63cb423c032ca06fbdc8efc73b66f4a",
|
| 216 |
+
"images": [
|
| 217 |
+
{
|
| 218 |
+
"name": "CAM_FRONT",
|
| 219 |
+
"path": "code/python/datasets/sample4/frames/sample_001/cameras/CAM_FRONT.jpg",
|
| 220 |
+
"sha256": "d5624fcecf12310c3d0760362e097c38cdcdce987e032aaf2883071cd14e2732",
|
| 221 |
+
"bytes": 146297
|
| 222 |
+
},
|
| 223 |
+
{
|
| 224 |
+
"name": "CAM_FRONT_RIGHT",
|
| 225 |
+
"path": "code/python/datasets/sample4/frames/sample_001/cameras/CAM_FRONT_RIGHT.jpg",
|
| 226 |
+
"sha256": "dd64d7463bc06bfd5d5378bdd494b40d68d0894307f3ed971eaaaf131022cf68",
|
| 227 |
+
"bytes": 143628
|
| 228 |
+
},
|
| 229 |
+
{
|
| 230 |
+
"name": "CAM_FRONT_LEFT",
|
| 231 |
+
"path": "code/python/datasets/sample4/frames/sample_001/cameras/CAM_FRONT_LEFT.jpg",
|
| 232 |
+
"sha256": "0fe15c0581db41e084df27acb2183b0ab9b0fd36da136ef1b4bcbda6f1a84c58",
|
| 233 |
+
"bytes": 146244
|
| 234 |
+
},
|
| 235 |
+
{
|
| 236 |
+
"name": "CAM_BACK",
|
| 237 |
+
"path": "code/python/datasets/sample4/frames/sample_001/cameras/CAM_BACK.jpg",
|
| 238 |
+
"sha256": "fdbdaa77e60ed5a6a762a4ea9a8eb1cb2ca9a3d94c610a55a2399a05c04b6ba6",
|
| 239 |
+
"bytes": 109601
|
| 240 |
+
},
|
| 241 |
+
{
|
| 242 |
+
"name": "CAM_BACK_LEFT",
|
| 243 |
+
"path": "code/python/datasets/sample4/frames/sample_001/cameras/CAM_BACK_LEFT.jpg",
|
| 244 |
+
"sha256": "000f7943769e91ffa842a7dfd1bfc31e0c3f4b37325a98846e8261304028f2ad",
|
| 245 |
+
"bytes": 112425
|
| 246 |
+
},
|
| 247 |
+
{
|
| 248 |
+
"name": "CAM_BACK_RIGHT",
|
| 249 |
+
"path": "code/python/datasets/sample4/frames/sample_001/cameras/CAM_BACK_RIGHT.jpg",
|
| 250 |
+
"sha256": "39d28a8a5a6f61370a7e7e1e965a6d8610b571195c50fd29da960e1fb2fbee80",
|
| 251 |
+
"bytes": 125712
|
| 252 |
+
}
|
| 253 |
+
]
|
| 254 |
+
}
|
| 255 |
+
}
|
| 256 |
+
},
|
| 257 |
+
"sample_002": {
|
| 258 |
+
"dataset_index": 2,
|
| 259 |
+
"sample_token": "c5f58c19249d4137ae063b0e9ecd8b8e",
|
| 260 |
+
"scene_token": "fcbccedd61424f1b85dcbf8f897f9754",
|
| 261 |
+
"prev_token": "3950bd41f74548429c0f7700ff3d8269",
|
| 262 |
+
"next_token": "700c1a25559b4433be532de3475e58a9",
|
| 263 |
+
"is_scene_start": false,
|
| 264 |
+
"assets": {
|
| 265 |
+
"can_bus": {
|
| 266 |
+
"dtype": "float16",
|
| 267 |
+
"shape": [
|
| 268 |
+
1,
|
| 269 |
+
18
|
| 270 |
+
],
|
| 271 |
+
"bytes": 36,
|
| 272 |
+
"path": "code/python/datasets/sample4/frames/sample_002/can_bus.raw",
|
| 273 |
+
"sha256": "e8badd418a0eb1905986c2a2a5e87fa6e16532dc4899cfff93f47dbe51bed9be"
|
| 274 |
+
},
|
| 275 |
+
"shift": {
|
| 276 |
+
"dtype": "float16",
|
| 277 |
+
"shape": [
|
| 278 |
+
1,
|
| 279 |
+
2
|
| 280 |
+
],
|
| 281 |
+
"bytes": 4,
|
| 282 |
+
"path": "code/python/datasets/sample4/frames/sample_002/shift.raw",
|
| 283 |
+
"sha256": "9cd32a1a7fbb67a9841ebee6acf497db1fa6142ec9e3ec5c9bb1596dd8be81e8"
|
| 284 |
+
},
|
| 285 |
+
"lidar2img": {
|
| 286 |
+
"dtype": "float16",
|
| 287 |
+
"shape": [
|
| 288 |
+
1,
|
| 289 |
+
6,
|
| 290 |
+
4,
|
| 291 |
+
4
|
| 292 |
+
],
|
| 293 |
+
"bytes": 192,
|
| 294 |
+
"path": "code/python/datasets/sample4/frames/sample_002/lidar2img.raw",
|
| 295 |
+
"sha256": "7d524fefbeec4c1014ab2773b722aaa13774cb4787fb950e27b1b4de7cc3d403"
|
| 296 |
+
},
|
| 297 |
+
"rotation_can_bus": {
|
| 298 |
+
"dtype": "float32",
|
| 299 |
+
"shape": [
|
| 300 |
+
1,
|
| 301 |
+
18
|
| 302 |
+
],
|
| 303 |
+
"bytes": 72,
|
| 304 |
+
"path": "code/python/datasets/sample4/frames/sample_002/rotation_can_bus.raw",
|
| 305 |
+
"sha256": "a08e149204cd22c460e8b5cc62726de1b29d22fb05b6c33b450e1f94a162d3c1"
|
| 306 |
+
},
|
| 307 |
+
"camera_images": {
|
| 308 |
+
"type": "raw_camera_jpeg_multiview",
|
| 309 |
+
"order": [
|
| 310 |
+
"CAM_FRONT",
|
| 311 |
+
"CAM_FRONT_RIGHT",
|
| 312 |
+
"CAM_FRONT_LEFT",
|
| 313 |
+
"CAM_BACK",
|
| 314 |
+
"CAM_BACK_LEFT",
|
| 315 |
+
"CAM_BACK_RIGHT"
|
| 316 |
+
],
|
| 317 |
+
"decode": "cv2.imread BGR",
|
| 318 |
+
"pipeline": "BGR -> RGB -> normalize(mean/std) -> resize(800,450) -> CHW -> pad to 480 rows",
|
| 319 |
+
"mean": [
|
| 320 |
+
123.675,
|
| 321 |
+
116.28,
|
| 322 |
+
103.53
|
| 323 |
+
],
|
| 324 |
+
"std": [
|
| 325 |
+
58.395,
|
| 326 |
+
57.12,
|
| 327 |
+
57.375
|
| 328 |
+
],
|
| 329 |
+
"to_rgb": true,
|
| 330 |
+
"resize": [
|
| 331 |
+
800,
|
| 332 |
+
450
|
| 333 |
+
],
|
| 334 |
+
"output_shape_unpadded": [
|
| 335 |
+
6,
|
| 336 |
+
3,
|
| 337 |
+
450,
|
| 338 |
+
800
|
| 339 |
+
],
|
| 340 |
+
"output_shape_padded": [
|
| 341 |
+
6,
|
| 342 |
+
3,
|
| 343 |
+
480,
|
| 344 |
+
800
|
| 345 |
+
],
|
| 346 |
+
"reference_preprocessed_sha256": "97fc26501e167d58279ce6229847cd6753b13b2b1795effba7dcbf282f99fc2e",
|
| 347 |
+
"images": [
|
| 348 |
+
{
|
| 349 |
+
"name": "CAM_FRONT",
|
| 350 |
+
"path": "code/python/datasets/sample4/frames/sample_002/cameras/CAM_FRONT.jpg",
|
| 351 |
+
"sha256": "0bb82b89f9e6ce47d9e1bcd747b01fe3fd342ccc67eb7a9c693a3fc3fc73a6bc",
|
| 352 |
+
"bytes": 145856
|
| 353 |
+
},
|
| 354 |
+
{
|
| 355 |
+
"name": "CAM_FRONT_RIGHT",
|
| 356 |
+
"path": "code/python/datasets/sample4/frames/sample_002/cameras/CAM_FRONT_RIGHT.jpg",
|
| 357 |
+
"sha256": "70856b9d713c279b4b3d5911be30944f339760cc81da5a8f2c4d90a20276c1d3",
|
| 358 |
+
"bytes": 132652
|
| 359 |
+
},
|
| 360 |
+
{
|
| 361 |
+
"name": "CAM_FRONT_LEFT",
|
| 362 |
+
"path": "code/python/datasets/sample4/frames/sample_002/cameras/CAM_FRONT_LEFT.jpg",
|
| 363 |
+
"sha256": "4782aaa9180ef1fe153e702031a3249ada09bdb056ad13ff69abaa2f9e59e35a",
|
| 364 |
+
"bytes": 145095
|
| 365 |
+
},
|
| 366 |
+
{
|
| 367 |
+
"name": "CAM_BACK",
|
| 368 |
+
"path": "code/python/datasets/sample4/frames/sample_002/cameras/CAM_BACK.jpg",
|
| 369 |
+
"sha256": "a9a05b52aea16ccf730447170eb836697c26928a112a74249caf423d91d28306",
|
| 370 |
+
"bytes": 111521
|
| 371 |
+
},
|
| 372 |
+
{
|
| 373 |
+
"name": "CAM_BACK_LEFT",
|
| 374 |
+
"path": "code/python/datasets/sample4/frames/sample_002/cameras/CAM_BACK_LEFT.jpg",
|
| 375 |
+
"sha256": "a28438e9ff68ccab2ce3a8da4151591a2b15780aecdc21185a911318ed923f72",
|
| 376 |
+
"bytes": 112445
|
| 377 |
+
},
|
| 378 |
+
{
|
| 379 |
+
"name": "CAM_BACK_RIGHT",
|
| 380 |
+
"path": "code/python/datasets/sample4/frames/sample_002/cameras/CAM_BACK_RIGHT.jpg",
|
| 381 |
+
"sha256": "1578ba72afd2cc91abb1295c3404a432e41ffc7385b4031a43b586acfb693872",
|
| 382 |
+
"bytes": 132515
|
| 383 |
+
}
|
| 384 |
+
]
|
| 385 |
+
}
|
| 386 |
+
}
|
| 387 |
+
},
|
| 388 |
+
"sample_003": {
|
| 389 |
+
"dataset_index": 3,
|
| 390 |
+
"sample_token": "700c1a25559b4433be532de3475e58a9",
|
| 391 |
+
"scene_token": "fcbccedd61424f1b85dcbf8f897f9754",
|
| 392 |
+
"prev_token": "c5f58c19249d4137ae063b0e9ecd8b8e",
|
| 393 |
+
"next_token": "747aa46b9a4641fe90db05d97db2acea",
|
| 394 |
+
"is_scene_start": false,
|
| 395 |
+
"assets": {
|
| 396 |
+
"can_bus": {
|
| 397 |
+
"dtype": "float16",
|
| 398 |
+
"shape": [
|
| 399 |
+
1,
|
| 400 |
+
18
|
| 401 |
+
],
|
| 402 |
+
"bytes": 36,
|
| 403 |
+
"path": "code/python/datasets/sample4/frames/sample_003/can_bus.raw",
|
| 404 |
+
"sha256": "dbc4945c65d5804131f75f04578e3fbc288330ff5fa977f224eda07eb96e463a"
|
| 405 |
+
},
|
| 406 |
+
"shift": {
|
| 407 |
+
"dtype": "float16",
|
| 408 |
+
"shape": [
|
| 409 |
+
1,
|
| 410 |
+
2
|
| 411 |
+
],
|
| 412 |
+
"bytes": 4,
|
| 413 |
+
"path": "code/python/datasets/sample4/frames/sample_003/shift.raw",
|
| 414 |
+
"sha256": "8cc4c98a9615a420d40104ae51d4f1f50b719a9cfec4be8d1b65b4b704517def"
|
| 415 |
+
},
|
| 416 |
+
"lidar2img": {
|
| 417 |
+
"dtype": "float16",
|
| 418 |
+
"shape": [
|
| 419 |
+
1,
|
| 420 |
+
6,
|
| 421 |
+
4,
|
| 422 |
+
4
|
| 423 |
+
],
|
| 424 |
+
"bytes": 192,
|
| 425 |
+
"path": "code/python/datasets/sample4/frames/sample_003/lidar2img.raw",
|
| 426 |
+
"sha256": "64cd7874438918c64f54414d1b873b0b0fcf05b2b601f2014c4fe5a8316e7a2c"
|
| 427 |
+
},
|
| 428 |
+
"rotation_can_bus": {
|
| 429 |
+
"dtype": "float32",
|
| 430 |
+
"shape": [
|
| 431 |
+
1,
|
| 432 |
+
18
|
| 433 |
+
],
|
| 434 |
+
"bytes": 72,
|
| 435 |
+
"path": "code/python/datasets/sample4/frames/sample_003/rotation_can_bus.raw",
|
| 436 |
+
"sha256": "3c88129317e663a14f4ba55a3c0c3cc25e9ec2cd4c497276567e4ecfed8a9ab6"
|
| 437 |
+
},
|
| 438 |
+
"camera_images": {
|
| 439 |
+
"type": "raw_camera_jpeg_multiview",
|
| 440 |
+
"order": [
|
| 441 |
+
"CAM_FRONT",
|
| 442 |
+
"CAM_FRONT_RIGHT",
|
| 443 |
+
"CAM_FRONT_LEFT",
|
| 444 |
+
"CAM_BACK",
|
| 445 |
+
"CAM_BACK_LEFT",
|
| 446 |
+
"CAM_BACK_RIGHT"
|
| 447 |
+
],
|
| 448 |
+
"decode": "cv2.imread BGR",
|
| 449 |
+
"pipeline": "BGR -> RGB -> normalize(mean/std) -> resize(800,450) -> CHW -> pad to 480 rows",
|
| 450 |
+
"mean": [
|
| 451 |
+
123.675,
|
| 452 |
+
116.28,
|
| 453 |
+
103.53
|
| 454 |
+
],
|
| 455 |
+
"std": [
|
| 456 |
+
58.395,
|
| 457 |
+
57.12,
|
| 458 |
+
57.375
|
| 459 |
+
],
|
| 460 |
+
"to_rgb": true,
|
| 461 |
+
"resize": [
|
| 462 |
+
800,
|
| 463 |
+
450
|
| 464 |
+
],
|
| 465 |
+
"output_shape_unpadded": [
|
| 466 |
+
6,
|
| 467 |
+
3,
|
| 468 |
+
450,
|
| 469 |
+
800
|
| 470 |
+
],
|
| 471 |
+
"output_shape_padded": [
|
| 472 |
+
6,
|
| 473 |
+
3,
|
| 474 |
+
480,
|
| 475 |
+
800
|
| 476 |
+
],
|
| 477 |
+
"reference_preprocessed_sha256": "da157bc039b66254cb57f827adcdfde783d324a078a15af9740a3701697eba15",
|
| 478 |
+
"images": [
|
| 479 |
+
{
|
| 480 |
+
"name": "CAM_FRONT",
|
| 481 |
+
"path": "code/python/datasets/sample4/frames/sample_003/cameras/CAM_FRONT.jpg",
|
| 482 |
+
"sha256": "54384bc8c4896f349bdc643b573f398fbb03ee842ac2af2af2c602a62f48f79d",
|
| 483 |
+
"bytes": 147138
|
| 484 |
+
},
|
| 485 |
+
{
|
| 486 |
+
"name": "CAM_FRONT_RIGHT",
|
| 487 |
+
"path": "code/python/datasets/sample4/frames/sample_003/cameras/CAM_FRONT_RIGHT.jpg",
|
| 488 |
+
"sha256": "4c5209d28a7e3a5f29bd7c02c29bd0f53c669afc43b17af2db098371a9711dc1",
|
| 489 |
+
"bytes": 145853
|
| 490 |
+
},
|
| 491 |
+
{
|
| 492 |
+
"name": "CAM_FRONT_LEFT",
|
| 493 |
+
"path": "code/python/datasets/sample4/frames/sample_003/cameras/CAM_FRONT_LEFT.jpg",
|
| 494 |
+
"sha256": "1f20956c425bdc84da1bf2610b7214d48930652782394aa0f1f3bef94ce0be35",
|
| 495 |
+
"bytes": 146337
|
| 496 |
+
},
|
| 497 |
+
{
|
| 498 |
+
"name": "CAM_BACK",
|
| 499 |
+
"path": "code/python/datasets/sample4/frames/sample_003/cameras/CAM_BACK.jpg",
|
| 500 |
+
"sha256": "34e84e10e04ab229864bd0ddd3067442c0b58e249eeb48088ab417cb259b4985",
|
| 501 |
+
"bytes": 112516
|
| 502 |
+
},
|
| 503 |
+
{
|
| 504 |
+
"name": "CAM_BACK_LEFT",
|
| 505 |
+
"path": "code/python/datasets/sample4/frames/sample_003/cameras/CAM_BACK_LEFT.jpg",
|
| 506 |
+
"sha256": "cdf1f064c61add58e17d7011713aebbe40be9d7e8290c8816f585fca3841e033",
|
| 507 |
+
"bytes": 113884
|
| 508 |
+
},
|
| 509 |
+
{
|
| 510 |
+
"name": "CAM_BACK_RIGHT",
|
| 511 |
+
"path": "code/python/datasets/sample4/frames/sample_003/cameras/CAM_BACK_RIGHT.jpg",
|
| 512 |
+
"sha256": "eae354ba8407662caa6c50934b6ee306a4e5aaaeec0b0f2e40c2f6b481485d3f",
|
| 513 |
+
"bytes": 135629
|
| 514 |
+
}
|
| 515 |
+
]
|
| 516 |
+
}
|
| 517 |
+
}
|
| 518 |
+
}
|
| 519 |
+
},
|
| 520 |
+
"contract": {
|
| 521 |
+
"image_pipeline": "cv2.imread BGR -> RGB -> normalize(mean=[123.675,116.28,103.53], std=[58.395,57.12,57.375]) -> cv2.resize(800,450) -> CHW -> zero-pad to (6,3,480,800)",
|
| 522 |
+
"camera_order": [
|
| 523 |
+
"CAM_FRONT",
|
| 524 |
+
"CAM_FRONT_RIGHT",
|
| 525 |
+
"CAM_FRONT_LEFT",
|
| 526 |
+
"CAM_BACK",
|
| 527 |
+
"CAM_BACK_LEFT",
|
| 528 |
+
"CAM_BACK_RIGHT"
|
| 529 |
+
],
|
| 530 |
+
"model_path": "W8A8 backbone + scene-start/temporal encoder + decoder AidLite QNN contexts",
|
| 531 |
+
"temporal_state": "scene-start uses reset semantics; temporal frames use previous live bev_embed as prev_bev",
|
| 532 |
+
"scene_start_inputs": "camera_images + can_bus + lidar2img; no prev_bev/shift input is required for scene-start encoder",
|
| 533 |
+
"temporal_inputs": "camera_images + can_bus + lidar2img + shift + rotated previous live bev_embed"
|
| 534 |
+
}
|
| 535 |
+
}
|
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_000/cameras/CAM_BACK.jpg
ADDED
|
Git LFS Details
|
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_000/cameras/CAM_BACK_LEFT.jpg
ADDED
|
Git LFS Details
|
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_000/cameras/CAM_BACK_RIGHT.jpg
ADDED
|
Git LFS Details
|
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_000/cameras/CAM_FRONT.jpg
ADDED
|
Git LFS Details
|
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_000/cameras/CAM_FRONT_LEFT.jpg
ADDED
|
Git LFS Details
|
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_000/cameras/CAM_FRONT_RIGHT.jpg
ADDED
|
Git LFS Details
|
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_000/can_bus.raw
ADDED
|
Binary file (36 Bytes). View file
|
|
|
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_000/lidar2img.raw
ADDED
|
Binary file (192 Bytes). View file
|
|
|
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/cameras/CAM_BACK.jpg
ADDED
|
Git LFS Details
|
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/cameras/CAM_BACK_LEFT.jpg
ADDED
|
Git LFS Details
|
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/cameras/CAM_BACK_RIGHT.jpg
ADDED
|
Git LFS Details
|
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/cameras/CAM_FRONT.jpg
ADDED
|
Git LFS Details
|
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/cameras/CAM_FRONT_LEFT.jpg
ADDED
|
Git LFS Details
|
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/cameras/CAM_FRONT_RIGHT.jpg
ADDED
|
Git LFS Details
|
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/can_bus.raw
ADDED
|
Binary file (36 Bytes). View file
|
|
|
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/lidar2img.raw
ADDED
|
Binary file (192 Bytes). View file
|
|
|
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/rotation_can_bus.raw
ADDED
|
Binary file (72 Bytes). View file
|
|
|
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_001/shift.raw
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
��T)
|
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/cameras/CAM_BACK.jpg
ADDED
|
Git LFS Details
|
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/cameras/CAM_BACK_LEFT.jpg
ADDED
|
Git LFS Details
|
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/cameras/CAM_BACK_RIGHT.jpg
ADDED
|
Git LFS Details
|
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/cameras/CAM_FRONT.jpg
ADDED
|
Git LFS Details
|
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/cameras/CAM_FRONT_LEFT.jpg
ADDED
|
Git LFS Details
|
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/cameras/CAM_FRONT_RIGHT.jpg
ADDED
|
Git LFS Details
|
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/can_bus.raw
ADDED
|
Binary file (36 Bytes). View file
|
|
|
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/lidar2img.raw
ADDED
|
Binary file (192 Bytes). View file
|
|
|
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/rotation_can_bus.raw
ADDED
|
Binary file (72 Bytes). View file
|
|
|
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_002/shift.raw
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
>�I)
|
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_003/cameras/CAM_BACK.jpg
ADDED
|
Git LFS Details
|
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_003/cameras/CAM_BACK_LEFT.jpg
ADDED
|
Git LFS Details
|
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_003/cameras/CAM_BACK_RIGHT.jpg
ADDED
|
Git LFS Details
|
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_003/cameras/CAM_FRONT.jpg
ADDED
|
Git LFS Details
|
targets/QCS8550/FP16/code/python/datasets/sample4/frames/sample_003/cameras/CAM_FRONT_LEFT.jpg
ADDED
|
Git LFS Details
|