File size: 4,075 Bytes
6dcc0f2 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 | ## Model Information
### Source model
- Input shape: [[1,6,3,480,800]]
- Number of parameters: 33.7M
- Model size: 128.5 FP32
- Output shape: boxes [[300,9]], scores [[300]], labels [[300]]
Source model repository: [BEVFormer](https://github.com/fundamentalvision/bevformer)
## Inference with AidLite SDK
### SDK installation
Model Farm uses AidLite SDK as the model inference SDK. For details, please refer to the [AidLite Developer Documentation](https://docs.aidlux.com/software/ai-sdk/aidlite_guide)
```bash
sudo aid-pkg update
sudo aid-pkg install aidlite-sdk
sudo aid-pkg install aidlite-{QNN VERSION}
```
Verify AidLite SDK:
```bash
python3 -c "import aidlite ; print(aidlite.get_library_version())"
python3 -c "import aidlite ; print(aidlite.get_py_library_version())"
```
### Environment Dependencies
Except for AidLite SDK, this BEVFormer demo also requires the following board-side Python environment.
Verified environment:
```bash
Python 3.10.12
numpy 1.26.4
opencv-python / cv2 4.13.0
Pillow / PIL 10.4.0
```
Required Python packages:
```bash
numpy
opencv-python
Pillow
```
Quick check:
```bash
python3 -c "import numpy, cv2; from PIL import Image; print('python env ok')"
```
The demo supports model directories for QCS8550, IQ9, QCS8625, and IQ8. A normal host or development container can only run dry-run package checks and cannot execute real QNN inference without AidLite.
## Model Directory Rule
Model SHA verification is always enabled. The platform is inferred from the model path, so no extra SHA option is required.
Supported model directories:
```bash
./models/QCS8550/FP16
./models/IQ9/FP16
./models/QCS8625/FP16
./models/IQ8/FP16
```
Platform mapping:
```text
QCS8550 -> QCS8550 model SHA table
IQ9 -> QCS8550 model SHA table
QCS8625 -> QCS8625 model SHA table
IQ8 -> QCS8625 model SHA table
```
Each platform directory must contain all four context files:
```bash
backbone_context.bin.aidem
scene_start_encoder_context.bin.aidem
temporal_encoder_context.bin.aidem
decoder_context.bin.aidem
```
If all four `--*_model` arguments are provided, the model paths decide the platform. All four paths must belong to the same platform directory. If model arguments are omitted, the script uses the first complete directory found in this order:
```text
QCS8550, IQ9, QCS8625, IQ8
```
## Run Demo
Basic command:
```bash
cd /home/aidlux/bevformer_delivery_demo_2026_07_08
python3 code/python/run_test.py --invoke_nums 4 --output_dir ./outputs/final_sample4
```
Full command with explicit model paths and dataset manifest:
```bash
python3 code/python/run_test.py \
--backbone_model ./models/QCS8625/FP16/backbone_context.bin.aidem \
--scene_start_encoder_model ./models/QCS8625/FP16/scene_start_encoder_context.bin.aidem \
--encoder_model ./models/QCS8625/FP16/temporal_encoder_context.bin.aidem \
--decoder_model ./models/QCS8625/FP16/decoder_context.bin.aidem \
--asset_manifest ./code/python/datasets/sample4/asset_manifest.json \
--invoke_nums 4 \
--output_dir ./outputs/final_sample4_qcs8625
```
The basic command auto-discovers the first complete model directory under `models/<platform>/FP16`. For QCS8550, IQ9, or IQ8, replace `QCS8625` in the full command with `QCS8550`, `IQ9`, or `IQ8`. The four model paths must use the same platform directory.
Dry-run package check:
```bash
python3 code/python/run_test.py --dry_run --check_raw_assets
```
The demo input is specified by:
```bash
./code/python/datasets/sample4/asset_manifest.json
```
The manifest describes four continuous BEVFormer frames. Each frame contains six raw camera JPG images and the required calibration / temporal auxiliary tensors. The demo starts from original six-camera images, not preprocessed image tensors.
Frame execution order:
```bash
frame 000: scene-start encoder
frame 001: temporal encoder
frame 002: temporal encoder
frame 003: temporal encoder
```
Main output files include:
```bash
bevformer_demo_summary.json
run.log
frameXXX_final_coordinates.npz
frameXXX_camera_grid.png
sample4_camera_grid.gif
```
|