File size: 4,367 Bytes
ebe01a0 723941c ebe01a0 7697b79 328b085 7697b79 ebe01a0 723941c ebe01a0 723941c ebe01a0 723941c ebe01a0 4fc61de ebe01a0 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 | ---
license: other
tags:
- heal
- horizon
- bev
---
# BEV Multitask + HENet-tiny
A camera-only BEV multitask model for 3D detection, occupancy, stop-line detection, and online lane-level mapping.
BEV Multitask + HENet-tiny lifts six surround cameras into a shared bird’s-eye feature and runs five heads together—dynamic/static objects, occupancy, stop lines, and vectorized map—so scene cues stay consistent. This release is calibrated for Horizon J6M / J6P deployment without lidar at inference.
---
## Deployment Metrics
### Model Parameters
| Model | Model Input | Backbone | Neck | Model Output |
|---|---|---|---|---|
| BEVFusionMultitask | 6-camera multi-view images `(B,6,3,512,960)` | HENet-tiny | FPN + SingleBevFormerViewTransformer | dynamic/static 3D boxes `(B,N,cls+reg)`; occ voxel semantics; discrete-map 2D boxes; map polylines |
### Accuracy Metrics
| March | Metric | float | calibration | hbm |
| --- | --- | --- | --- | --- |
| J6M | Dynamic NDS | 0.4553 | 0.4480 | 0.4535 |
| | Static NDS | 0.4873 | 0.4839 | 0.4842 |
| | Occ mIoU | 0.3422 | 0.3401 | 0.3399 |
| | stop_line NDS | 0.4802 | 0.4762 | 0.4759 |
| | MapTR mAP | 0.5486 | 0.5405 | 0.5376 |
> Results measured with `march = March.NASH_M` (J6M) configuration; this task has no QAT stage.
>
> HEAL version: heal 0.14.0 / hbdk4-compiler 4.13.1 / horizon_plugin_pytorch 3.4.5 (board UCP 3.16.2).
### Performance Metrics
> **Performance benchmark**: FPS is measured with single-core 8 threads; latency is single-core single-thread; memory is peak DDR usage.
> Data source: `tools/test_perf_all.py` (default sampling interval `-d 200ms`, perf-time 1).
| March | latency (ms) | fps | Memory Usage (MB) |
|---|---|---|---|
| J6M | 77.82 | 13.21 | 386.90 |
| J6P | 47.65 | 76.17 | 373.50 |
---
## Model Overview
### Core Design
Six surround cameras are encoded by HENet-tiny, then lifted into a compact BEV canvas (about 100 m around the ego) with a single-frame BevFormer view transformer. That shared BEV feature fans out to five lightweight heads: SparseBEV-style queries for dynamic and static 3D boxes, a voxel occupancy head, a stop-line detector, and a MapTR-style polyline head for dividers, crossings, and boundaries — so detection, free space, and map cues stay consistent without running five separate networks.
- **Task type**: BEV multitask perception (dynamic det + static det + occupancy + discrete map + online map).
- **backbone**: HENet-tiny (`type=HENet`, `block_nums=[4,3,8,6]`, `embed_dims=[64,128,192,384]`, `GroupDWCB`/`AltDWCB`/`DWCB` blocks, `S2DDown` downsampling), initialized from the ImageNet-pretrained `henet_tinym` float checkpoint.
- **neck**: `FPN` (`in_strides=[32]` → `out_strides=[32]`, `_dim_=256`) + `BevFuseModule` (`input_c=384+256` → `fuse_c=256`; lidar branch disabled, `lidar_input=False`).
- **Dynamic det head**: `DenseBEVSparseHead` (8 classes, `num_anchor=384`, `num_decoder=6`, `MemoryBank` with `num_temp_instances=128` for tracking).
- **Static det head**: `DenseBEVSparseHead` (2 classes: `traffic_cone`, `barrier`; `num_anchor=200`).
- **Occupancy head**: `BevformerOccDetDecoder` + `BEVOCCHead2D` (`Dz=16`, 18 classes).
- **Discrete map head**: `BEVFormerDet2DDecoder` (1 class `stop_line`, `num_query=100`).
- **Online map head**: `MaptrDetDecoder` + `MapInstanceDetectorHead` (3 classes: `divider`, `ped_crossing`, `boundary`; 20 points per polyline).
- **Loss functions**: det — `FocalLoss` (cls) + `L1Loss` (reg) + `CrossEntropyLoss` (cns) + `GaussianFocalLoss` (yns); occ — `CrossEntropyLoss`; maptr — `MapTRCriterion`; discrete map — `BevFormer2DCriterion`.
- **Model input**: 6-camera multi-view images `(B,6,3,512,960)` (YUV444 + `Normalize(128,128)`; camera-only, no lidar at inference).
- **Model output**: dynamic/static 3D boxes + occ voxel semantics + stop-line boxes + map polylines.
### Official Repo and Paper
1. Horizon reference algorithm blog (Dense multitask_v3.0): https://developer.horizon.auto/blog/14189
2. SparseBEV-style det head repo: https://github.com/MCG-NJU/SparseBEV (paper: https://arxiv.org/abs/2308.09244)
3. MapTR repo: https://github.com/hustvl/MapTR (paper: https://arxiv.org/abs/2208.14437)
Note: The camera backbone HENet and the multitask assembly are HEAL in-house implementations; the official repos use different backbones and single-task setups.
|