source: one rebuild recipe per hosted AGPL file; yolov8n-plate provenance
Browse filesLicence files, notices, model cards and the source/ folder (AGPL Corresponding Source scripts) from the CamStack repository. Only files that differ from the Hub are included. No weight file is touched.
- README.md +1 -1
- THIRD_PARTY_MODELS.md +45 -24
- plateDetection/yolov8-plate/README.md +8 -8
- source/README.md +75 -15
- source/build-camstack-models.py +629 -224
README.md
CHANGED
|
@@ -77,7 +77,7 @@ marks an entry kept resolvable for a saved selection but no longer offered.
|
|
| 77 |
| `packageDetection/rfdetr-package` | `rfdetr-package` (default) | Roboflow RF-DETR | Apache-2.0 | [`Apache-2.0.txt`](LICENSES/Apache-2.0.txt) | RF-DETR-M fine-tuned to 2 classes at 576; exported to ONNX, OpenVINO IR and CoreML; training checkpoint .pth also published |
|
| 78 |
| `faceDetection/yunet` | `yunet-2023mar` (default) | OpenCV Zoo YuNet | MIT | [`MIT-yunet-reference.txt`](LICENSES/MIT-yunet-reference.txt), [`BSD-3-Clause.txt`](LICENSES/BSD-3-Clause.txt) | input contract baked into the graph (the pool RGB /255 becomes the upstream BGR 0..255: Mul(255) plus a channel Gather); internal tensor names cleared in the OpenVINO IR; ONNX fp32, OpenVINO IR fp32, CoreML fp16 (scripts/build-yunet-model.py) |
|
| 79 |
| `faceRecognition/auraface` | `auraface-r100` (default) | fal.ai AuraFace v1 (glintr100.onnx only) | Apache-2.0 | [`Apache-2.0.txt`](LICENSES/Apache-2.0.txt) | input scaling y = 2x - 1 baked ahead of the first Conv (scripts/build-auraface-model.py); converted to OpenVINO IR and CoreML FP16 |
|
| 80 |
-
| `plateDetection/yolov8-plate` | `yolov8n-plate` (default) |
|
| 81 |
| `plateRecognition/cct-s-v2-global` | `cct-s-v2-global` (default) | ankandrew/fast-plate-ocr | MIT | [`MIT-fast-plate-ocr.txt`](LICENSES/MIT-fast-plate-ocr.txt) | input head removed (recorded in the ONNX doc_string); converted to OpenVINO IR and CoreML |
|
| 82 |
| `animalClassification/animal-classifier` | `animal-classifier` (default) | timm EfficientNet-Lite0, fine-tuned by CamStack | Apache-2.0 | [`Apache-2.0.txt`](LICENSES/Apache-2.0.txt) | fine-tuned by CamStack on the Open Images V7 animal subset (8 classes); exported to ONNX, OpenVINO IR and CoreML |
|
| 83 |
| `animalClassification/bird-classifier` | `bird-classifier` (default) | Google AIY Vision birds_V1 | Apache-2.0 | [`Apache-2.0.txt`](LICENSES/Apache-2.0.txt) | converted from TensorFlow with tf2onnx; OpenVINO IR and CoreML derived from it |
|
|
|
|
| 77 |
| `packageDetection/rfdetr-package` | `rfdetr-package` (default) | Roboflow RF-DETR | Apache-2.0 | [`Apache-2.0.txt`](LICENSES/Apache-2.0.txt) | RF-DETR-M fine-tuned to 2 classes at 576; exported to ONNX, OpenVINO IR and CoreML; training checkpoint .pth also published |
|
| 78 |
| `faceDetection/yunet` | `yunet-2023mar` (default) | OpenCV Zoo YuNet | MIT | [`MIT-yunet-reference.txt`](LICENSES/MIT-yunet-reference.txt), [`BSD-3-Clause.txt`](LICENSES/BSD-3-Clause.txt) | input contract baked into the graph (the pool RGB /255 becomes the upstream BGR 0..255: Mul(255) plus a channel Gather); internal tensor names cleared in the OpenVINO IR; ONNX fp32, OpenVINO IR fp32, CoreML fp16 (scripts/build-yunet-model.py) |
|
| 79 |
| `faceRecognition/auraface` | `auraface-r100` (default) | fal.ai AuraFace v1 (glintr100.onnx only) | Apache-2.0 | [`Apache-2.0.txt`](LICENSES/Apache-2.0.txt) | input scaling y = 2x - 1 baked ahead of the first Conv (scripts/build-auraface-model.py); converted to OpenVINO IR and CoreML FP16 |
|
| 80 |
+
| `plateDetection/yolov8-plate` | `yolov8n-plate` (default) | ml-debi/yolov8-license-plate-detection (ONNX, OpenVINO) and Koushim/yolov8-license-plate-detection (CoreML), YOLOv8n fine-tunes | AGPL-3.0 | [`AGPL-3.0.txt`](LICENSES/AGPL-3.0.txt) | ONNX redistributed unmodified; converted to OpenVINO IR from the ONNX, and exported to CoreML and TFLite from the checkpoint (scripts/build-camstack-models.py) |
|
| 81 |
| `plateRecognition/cct-s-v2-global` | `cct-s-v2-global` (default) | ankandrew/fast-plate-ocr | MIT | [`MIT-fast-plate-ocr.txt`](LICENSES/MIT-fast-plate-ocr.txt) | input head removed (recorded in the ONNX doc_string); converted to OpenVINO IR and CoreML |
|
| 82 |
| `animalClassification/animal-classifier` | `animal-classifier` (default) | timm EfficientNet-Lite0, fine-tuned by CamStack | Apache-2.0 | [`Apache-2.0.txt`](LICENSES/Apache-2.0.txt) | fine-tuned by CamStack on the Open Images V7 animal subset (8 classes); exported to ONNX, OpenVINO IR and CoreML |
|
| 83 |
| `animalClassification/bird-classifier` | `bird-classifier` (default) | Google AIY Vision birds_V1 | Apache-2.0 | [`Apache-2.0.txt`](LICENSES/Apache-2.0.txt) | converted from TensorFlow with tf2onnx; OpenVINO IR and CoreML derived from it |
|
THIRD_PARTY_MODELS.md
CHANGED
|
@@ -36,7 +36,7 @@ under [Non-commercial models](#non-commercial-models).
|
|
| 36 |
- RF-DETR © Roboflow, Apache-2.0. https://github.com/roboflow/rf-detr
|
| 37 |
- YuNet weights Copyright (c) 2020 Shiqi Yu, MIT. Training code: ShiqiYu/libfacedetection.train, BSD-3-Clause. https://github.com/opencv/opencv_zoo/tree/main/models/face_detection_yunet
|
| 38 |
- AuraFace v1 by fal.ai, Apache-2.0. https://huggingface.co/fal/AuraFace-v1
|
| 39 |
-
- © Ultralytics, AGPL-3.0 (YOLOv8n base
|
| 40 |
- fast-plate-ocr, MIT License, copyright (c) 2024 ankandrew (full notice: licenses/models/MIT-fast-plate-ocr.txt). https://github.com/ankandrew/fast-plate-ocr
|
| 41 |
- EfficientNet-Lite0 via timm (Ross Wightman), Apache-2.0. Open Images V7 annotations © Google LLC, CC BY 4.0; images listed by Google as CC BY 2.0, without warranty. https://github.com/huggingface/pytorch-image-models
|
| 42 |
- torchvision MobileNetV3, BSD-3-Clause. Trained on the Vehicle classification dataset by DrBimmer (https://huggingface.co/datasets/DrBimmer/vehicle-classification), CC BY-NC 4.0. https://github.com/pytorch/vision
|
|
@@ -68,28 +68,49 @@ is published in the mirror itself, in its **`source/`** folder,
|
|
| 68 |
**https://huggingface.co/camstack/camstack-models/tree/main/source**, together
|
| 69 |
with the upstream checkpoints it starts from:
|
| 70 |
|
| 71 |
-
1. **The upstream checkpoints**
|
| 72 |
-
|
| 73 |
-
|
| 74 |
-
|
| 75 |
-
|
| 76 |
-
|
| 77 |
-
|
| 78 |
-
`
|
| 79 |
-
|
| 80 |
-
|
| 81 |
-
|
| 82 |
-
|
| 83 |
-
|
| 84 |
-
|
| 85 |
-
|
| 86 |
-
|
| 87 |
-
|
| 88 |
-
|
| 89 |
-
|
| 90 |
-
-
|
| 91 |
-
|
| 92 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 93 |
|
| 94 |
## Non-commercial models
|
| 95 |
|
|
@@ -138,7 +159,7 @@ entry kept resolvable for a saved selection but no longer offered.
|
|
| 138 |
|
| 139 |
| Catalog ids | Upstream | Code / weights | Training data | CamStack changes | Downloaded from |
|
| 140 |
| --- | --- | --- | --- | --- | --- |
|
| 141 |
-
| `yolov8n-plate` (default) | [
|
| 142 |
|
| 143 |
### Licence plate reading
|
| 144 |
|
|
|
|
| 36 |
- RF-DETR © Roboflow, Apache-2.0. https://github.com/roboflow/rf-detr
|
| 37 |
- YuNet weights Copyright (c) 2020 Shiqi Yu, MIT. Training code: ShiqiYu/libfacedetection.train, BSD-3-Clause. https://github.com/opencv/opencv_zoo/tree/main/models/face_detection_yunet
|
| 38 |
- AuraFace v1 by fal.ai, Apache-2.0. https://huggingface.co/fal/AuraFace-v1
|
| 39 |
+
- © Ultralytics, AGPL-3.0 (YOLOv8n base). Fine-tuned for licence plates by third parties: the ONNX file is best.onnx of https://huggingface.co/ml-debi/yolov8-license-plate-detection, redistributed unmodified (its author published no checkpoint); the CoreML and TFLite files are exported from best.pt of https://huggingface.co/Koushim/yolov8-license-plate-detection. Both cards say MIT; the Ultralytics base keeps them AGPL-3.0. Corresponding source: build-camstack-models.py at https://huggingface.co/camstack/camstack-models/tree/main/source. https://huggingface.co/ml-debi/yolov8-license-plate-detection
|
| 40 |
- fast-plate-ocr, MIT License, copyright (c) 2024 ankandrew (full notice: licenses/models/MIT-fast-plate-ocr.txt). https://github.com/ankandrew/fast-plate-ocr
|
| 41 |
- EfficientNet-Lite0 via timm (Ross Wightman), Apache-2.0. Open Images V7 annotations © Google LLC, CC BY 4.0; images listed by Google as CC BY 2.0, without warranty. https://github.com/huggingface/pytorch-image-models
|
| 42 |
- torchvision MobileNetV3, BSD-3-Clause. Trained on the Vehicle classification dataset by DrBimmer (https://huggingface.co/datasets/DrBimmer/vehicle-classification), CC BY-NC 4.0. https://github.com/pytorch/vision
|
|
|
|
| 68 |
**https://huggingface.co/camstack/camstack-models/tree/main/source**, together
|
| 69 |
with the upstream checkpoints it starts from:
|
| 70 |
|
| 71 |
+
1. **The upstream checkpoints.** Ultralytics publishes
|
| 72 |
+
`yolo26{n,s,m,l,x}.pt`, `yolov9{t,s,m,c}.pt` and `yolo26{n,s,m}-seg.pt` at
|
| 73 |
+
https://github.com/ultralytics/assets/releases. `yolov8n-plate` is two
|
| 74 |
+
third-party YOLOv8n licence-plate fine-tunes, neither trained by CamStack:
|
| 75 |
+
its ONNX file is `best.onnx` of
|
| 76 |
+
https://huggingface.co/ml-debi/yolov8-license-plate-detection (SHA-256
|
| 77 |
+
`85d23628…15bd50`, hosted unmodified), and its CoreML and TFLite files are
|
| 78 |
+
exported from `best.pt` of
|
| 79 |
+
https://huggingface.co/Koushim/yolov8-license-plate-detection. Both cards
|
| 80 |
+
say MIT; the Ultralytics base weights keep both AGPL-3.0, as their own
|
| 81 |
+
metadata states.
|
| 82 |
+
2. **The build script** in `source/`: `build-camstack-models.py`, with
|
| 83 |
+
`convert_lib.py` (the ONNX to OpenVINO / INT8 converter it calls, the same
|
| 84 |
+
file CamStack's Model Studio runs) and `requirements.txt`. It holds one
|
| 85 |
+
recipe per hosted file: the upstream checkpoint, the input size, the
|
| 86 |
+
Ultralytics `export()` arguments, the OpenVINO precision and the INT8
|
| 87 |
+
calibration count that file records about itself. Every hosted
|
| 88 |
+
Ultralytics-derived file is rebuilt by one invocation:
|
| 89 |
+
`build-camstack-models.py --variant <file stem>` (for example
|
| 90 |
+
`camstack-yolo26n-320` or `camstack-yolov9m-640`), `--family <folder>`
|
| 91 |
+
or `--all`; `--list` prints the recipes and `--check-hosted` confirms
|
| 92 |
+
against the mirror that every hosted file is a recipe output. The INT8
|
| 93 |
+
builds are NNCF post-training quantisation calibrated on COCO images: 128
|
| 94 |
+
for YOLO26, 300 for YOLOv9, as each IR records; the documented set is the
|
| 95 |
+
first N of COCO val2017 in file-name order. The Python
|
| 96 |
+
environment, the recorded tool versions and the calibration set are in
|
| 97 |
+
`source/README.md` and the script's header. These scripts are CamStack's
|
| 98 |
+
own code and are offered under the MIT licence; the weights they produce
|
| 99 |
+
keep their upstream licence.
|
| 100 |
+
|
| 101 |
+
Limits of a rebuild:
|
| 102 |
+
|
| 103 |
+
- **Tool versions.** With the Ultralytics, PyTorch, coremltools and OpenVINO
|
| 104 |
+
versions each recipe names, a rebuild reproduces the hosted weights. Other
|
| 105 |
+
versions give the same weights in a possibly different graph.
|
| 106 |
+
- **INT8 calibration.** An IR records how many images calibrated it, not
|
| 107 |
+
which ones. Calibrating on the same COCO images gives the documented build;
|
| 108 |
+
a different set gives an equivalent, not bit-identical, INT8 model.
|
| 109 |
+
- **`yolov8n-plate` ONNX has no checkpoint.** Its author published only the
|
| 110 |
+
ONNX file, so that file is the preferred form of the work that exists, and
|
| 111 |
+
CamStack redistributes it unmodified. CamStack's modification is the
|
| 112 |
+
OpenVINO conversion of it, which the script reproduces. The training data of
|
| 113 |
+
both plate fine-tunes is not published.
|
| 114 |
|
| 115 |
## Non-commercial models
|
| 116 |
|
|
|
|
| 159 |
|
| 160 |
| Catalog ids | Upstream | Code / weights | Training data | CamStack changes | Downloaded from |
|
| 161 |
| --- | --- | --- | --- | --- | --- |
|
| 162 |
+
| `yolov8n-plate` (default) | [ml-debi/yolov8-license-plate-detection (ONNX, OpenVINO) and Koushim/yolov8-license-plate-detection (CoreML), YOLOv8n fine-tunes](https://huggingface.co/ml-debi/yolov8-license-plate-detection) | AGPL-3.0 / AGPL-3.0 (`AGPL-3.0.txt`) | undisclosed licence-plate datasets of the two third-party fine-tunes (ml-debi, trained 2023-12-05; Koushim, trained 2025-05-25): UNKNOWN | ONNX redistributed unmodified; converted to OpenVINO IR from the ONNX, and exported to CoreML and TFLite from the checkpoint (scripts/build-camstack-models.py) | mirror `plateDetection/yolov8-plate` |
|
| 163 |
|
| 164 |
### Licence plate reading
|
| 165 |
|
plateDetection/yolov8-plate/README.md
CHANGED
|
@@ -1,17 +1,17 @@
|
|
| 1 |
-
<!-- Hand-written 2026-09-26 (plan docs/superpowers/plans/2026-09-26-model-license-compliance.md, Task 2) in camstack/server hf/camstack-models/. Edit it there, not on the Hub. -->
|
| 2 |
|
| 3 |
# plateDetection/yolov8-plate: YOLOv8n licence-plate detector
|
| 4 |
|
| 5 |
| | |
|
| 6 |
| --- | --- |
|
| 7 |
| Catalog id | `yolov8n-plate` |
|
| 8 |
-
| Upstream | Ultralytics YOLOv8n fine-tuned for licence plates by
|
| 9 |
-
| Obtained from |
|
| 10 |
| Code licence | AGPL-3.0 |
|
| 11 |
-
| **Weights licence** | **AGPL-3.0** (declared in
|
| 12 |
-
| Training data | **
|
| 13 |
-
| CamStack modifications |
|
| 14 |
-
| Corresponding source | [
|
| 15 |
-
| Commercial use | AGPL-3.0 obligations apply; training data
|
| 16 |
|
| 17 |
CamStack redistributes the files in this folder; every change from the upstream is stated above. Converting a model does not change its licence. The collection statement is [`LICENSE`](../../LICENSE); every model's attribution is in [`THIRD_PARTY_MODELS.md`](../../THIRD_PARTY_MODELS.md).
|
|
|
|
| 1 |
+
<!-- Hand-written 2026-09-26 (plan docs/superpowers/plans/2026-09-26-model-license-compliance.md, Task 2), provenance identified 2026-10-06, in camstack/server hf/camstack-models/. Edit it there, not on the Hub. -->
|
| 2 |
|
| 3 |
# plateDetection/yolov8-plate: YOLOv8n licence-plate detector
|
| 4 |
|
| 5 |
| | |
|
| 6 |
| --- | --- |
|
| 7 |
| Catalog id | `yolov8n-plate` |
|
| 8 |
+
| Upstream | Ultralytics YOLOv8n, fine-tuned for licence plates by two third parties: [ml-debi/yolov8-license-plate-detection](https://huggingface.co/ml-debi/yolov8-license-plate-detection) (ONNX, OpenVINO; trained 2023-12-05, Ultralytics 8.0.222) and [Koushim/yolov8-license-plate-detection](https://huggingface.co/Koushim/yolov8-license-plate-detection) (CoreML, TFLite; trained 2025-05-25, Ultralytics 8.3.144) |
|
| 9 |
+
| Obtained from | `onnx/`: ml-debi's `best.onnx`, byte-identical (SHA-256 `85d236280a1301ad98907947d284951dd2b20c23a6786ff50f7e6a8ec515bd50`). `coreml/`, `tflite/`: exported from Koushim's `best.pt` (SHA-256 `2d95861825bb4184404344c9cf809f40fd31dba785fe54e8ba5b9a3583789822`) |
|
| 10 |
| Code licence | AGPL-3.0 |
|
| 11 |
+
| **Weights licence** | **AGPL-3.0** (the Ultralytics base; declared in both models' metadata; [`LICENSES/AGPL-3.0.txt`](../../LICENSES/AGPL-3.0.txt)). Both upstream cards say MIT, which cannot lift the base's AGPL-3.0 |
|
| 12 |
+
| Training data | **not published** by either upstream |
|
| 13 |
+
| CamStack modifications | ONNX redistributed unmodified; OpenVINO IR converted from it; CoreML and TFLite (float32) exported from the checkpoint |
|
| 14 |
+
| Corresponding source | [`source/build-camstack-models.py`](../../source/build-camstack-models.py) `--variant camstack-yolov8n-plate` downloads both upstream files, verifies their hashes and rebuilds every file here. ml-debi published no checkpoint, so its ONNX is the only form of that model that exists. See the [source offer](../../THIRD_PARTY_MODELS.md#source-offer-for-the-agpl-30-weights) |
|
| 15 |
+
| Commercial use | AGPL-3.0 obligations apply; training data not published |
|
| 16 |
|
| 17 |
CamStack redistributes the files in this folder; every change from the upstream is stated above. Converting a model does not change its licence. The collection statement is [`LICENSE`](../../LICENSE); every model's attribution is in [`THIRD_PARTY_MODELS.md`](../../THIRD_PARTY_MODELS.md).
|
source/README.md
CHANGED
|
@@ -5,32 +5,92 @@
|
|
| 5 |
This folder is the **Corresponding Source** under the GNU AGPL-3.0 for the
|
| 6 |
Ultralytics-derived weights in this mirror (`objectDetection/yolo26`,
|
| 7 |
`objectDetection/yolov9`, `segmentation/yolo26-seg`, `plateDetection/yolov8-plate`).
|
| 8 |
-
The full source offer
|
| 9 |
[`THIRD_PARTY_MODELS.md`](../THIRD_PARTY_MODELS.md#source-offer-for-the-agpl-30-weights).
|
| 10 |
|
| 11 |
| File | What it is |
|
| 12 |
| --- | --- |
|
| 13 |
-
| `build-camstack-models.py` |
|
| 14 |
-
| `convert_lib.py` | The converter `build-camstack-models.py`
|
| 15 |
-
| `requirements.txt` | Python dependencies of `convert_lib.py`, with per-platform markers
|
| 16 |
-
| `export-models.py` |
|
| 17 |
|
| 18 |
## Rebuilding
|
| 19 |
|
| 20 |
```bash
|
| 21 |
python3 -m venv .venv && . .venv/bin/activate
|
| 22 |
-
pip install -r requirements.txt ultralytics
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 26 |
```
|
| 27 |
|
| 28 |
-
The
|
| 29 |
-
|
| 30 |
-
the
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 34 |
|
| 35 |
## Licences
|
| 36 |
|
|
|
|
| 5 |
This folder is the **Corresponding Source** under the GNU AGPL-3.0 for the
|
| 6 |
Ultralytics-derived weights in this mirror (`objectDetection/yolo26`,
|
| 7 |
`objectDetection/yolov9`, `segmentation/yolo26-seg`, `plateDetection/yolov8-plate`).
|
| 8 |
+
The full source offer is in
|
| 9 |
[`THIRD_PARTY_MODELS.md`](../THIRD_PARTY_MODELS.md#source-offer-for-the-agpl-30-weights).
|
| 10 |
|
| 11 |
| File | What it is |
|
| 12 |
| --- | --- |
|
| 13 |
+
| `build-camstack-models.py` | Rebuilds every file in the four folders above. One recipe per hosted file stem: upstream checkpoint, input size, the Ultralytics `export()` arguments, the OpenVINO precisions and the INT8 calibration count. Each value is what the hosted file records about itself in its metadata. |
|
| 14 |
+
| `convert_lib.py` | The converter `build-camstack-models.py` calls for OpenVINO: ONNX to IR (`ov.convert_model`, FP32 or FP16-compressed), NNCF INT8 post-training quantisation, and validation of every IR. It is the same file CamStack's Model Studio runs. |
|
| 15 |
+
| `requirements.txt` | Python dependencies of `convert_lib.py`, with per-platform markers. |
|
| 16 |
+
| `export-models.py` | An older bulk exporter for YOLOv8 / YOLOv9 / YOLO11 ids. No file in this mirror needs it. |
|
| 17 |
|
| 18 |
## Rebuilding
|
| 19 |
|
| 20 |
```bash
|
| 21 |
python3 -m venv .venv && . .venv/bin/activate
|
| 22 |
+
pip install -r requirements.txt ultralytics openvino nncf
|
| 23 |
+
python build-camstack-models.py --list # the recipes: one per hosted file stem
|
| 24 |
+
python build-camstack-models.py --check-hosted # every hosted file is a recipe output?
|
| 25 |
+
|
| 26 |
+
python build-camstack-models.py --variant camstack-yolo26n-320 # one hosted model, every format
|
| 27 |
+
python build-camstack-models.py --family yolov9 # one mirror folder
|
| 28 |
+
python build-camstack-models.py --all # the whole set
|
| 29 |
+
python build-camstack-models.py --all --formats onnx,openvino # some formats only
|
| 30 |
+
```
|
| 31 |
+
|
| 32 |
+
Results go to `./exported-models/<mirror path>`, for example
|
| 33 |
+
`exported-models/objectDetection/yolo26/openvino/camstack-yolo26n-320-int8.xml`.
|
| 34 |
+
Each build step runs in its own Python process: on macOS, CoreML, OpenVINO and
|
| 35 |
+
TensorFlow crash one another when they share one. CoreML needs macOS. TFLite
|
| 36 |
+
needs `tensorflow` and `onnx2tf`.
|
| 37 |
+
|
| 38 |
+
What the recipes encode (the hosted files record each of these):
|
| 39 |
+
|
| 40 |
+
| Folder | Files | ONNX | CoreML | OpenVINO IR |
|
| 41 |
+
| --- | --- | --- | --- | --- |
|
| 42 |
+
| `objectDetection/yolo26` | `camstack-yolo26{n,s,m,l,x}` @640 | static, opset 17 | static, batch 1 | FP16 as `<stem>.xml` and `<stem>-fp16.xml`; INT8 `<stem>-int8.xml` (128 images) |
|
| 43 |
+
| `objectDetection/yolo26` | `camstack-yolo26{n,s,m,l}-{320,256}` | dynamic axes, opset 17 | flexible batch 1..16, FP16 | FP32 `<stem>.xml`; INT8 `<stem>-int8.xml` (128 images) |
|
| 44 |
+
| `objectDetection/yolov9` | `camstack-yolov9{t,s,m,c}-{320,640}` | static, opset 13 | static, batch 1 | FP16 `<stem>-fp16.xml`; INT8 `<stem>-int8.xml` (300 images) |
|
| 45 |
+
| `segmentation/yolo26-seg` | `camstack-yolo26{n,s,m}-seg` @640 | static, opset 17 | static, batch 1 | FP16 `<stem>.xml` |
|
| 46 |
+
| `plateDetection/yolov8-plate` | `camstack-yolov8n-plate` @640 | third-party file, unmodified | static, batch 1 | FP16 `<stem>.xml` (from the ONNX); also TFLite float32 |
|
| 47 |
+
|
| 48 |
+
Recorded tool versions (Ultralytics / PyTorch / coremltools / OpenVINO):
|
| 49 |
+
YOLO26 @640 and YOLO26-seg 8.4.24 / 2.10.0 / 9.0 / 2026.2.1; YOLO26 @320 and
|
| 50 |
+
@256 8.4.92 / 2.13.0 for ONNX, 2.5.1 for CoreML / 8.3.0 / 2026.2.1; YOLOv9
|
| 51 |
+
8.4.98 / 2.13.0 / 9.0 / 2026.2.1; plate CoreML 8.4.24 / 2.10.0 / 9.0, plate IR
|
| 52 |
+
OpenVINO 2026.0.0. The script warns when the installed Ultralytics differs.
|
| 53 |
+
With these versions a rebuild gives the hosted weights. Other versions give the
|
| 54 |
+
same weights in a possibly different graph. The flexible-batch CoreML exports
|
| 55 |
+
need coremltools 8.3 with PyTorch 2.5.1, the macOS pins in `requirements.txt`;
|
| 56 |
+
coremltools 9 aborts on them.
|
| 57 |
+
|
| 58 |
+
### INT8 calibration
|
| 59 |
+
|
| 60 |
+
The INT8 IRs are NNCF post-training quantisation of the ONNX
|
| 61 |
+
(`convert_lib.quantize_openvino_int8`), calibrated on COCO images. The
|
| 62 |
+
documented calibration set is COCO val2017:
|
| 63 |
+
|
| 64 |
+
```bash
|
| 65 |
+
curl -LO http://images.cocodataset.org/zips/val2017.zip && unzip -q val2017.zip
|
| 66 |
+
python build-camstack-models.py --family yolo26 --calib-dir ./val2017
|
| 67 |
```
|
| 68 |
|
| 69 |
+
The recipe takes the first N images in file-name order: 128 for YOLO26, 300
|
| 70 |
+
for YOLOv9, as recorded in each IR's `rt_info`. Each image is letterboxed to
|
| 71 |
+
the model's input size with grey (114) padding and scaled to [0, 1], the way
|
| 72 |
+
CamStack feeds the model at runtime. `--calib-samples` overrides N. An IR does
|
| 73 |
+
not record WHICH images calibrated it, so a different set gives an equivalent
|
| 74 |
+
INT8 model, not a bit-identical one. Without calibration images the INT8 files
|
| 75 |
+
are skipped and the run says so.
|
| 76 |
+
|
| 77 |
+
### `yolov8n-plate`
|
| 78 |
+
|
| 79 |
+
Two third-party YOLOv8n licence-plate fine-tunes, neither trained by CamStack:
|
| 80 |
+
|
| 81 |
+
- `onnx/camstack-yolov8n-plate.onnx` is `best.onnx` of
|
| 82 |
+
[ml-debi/yolov8-license-plate-detection](https://huggingface.co/ml-debi/yolov8-license-plate-detection)
|
| 83 |
+
(revision `8062e8e8`, SHA-256 `85d236280a1301ad98907947d284951dd2b20c23a6786ff50f7e6a8ec515bd50`),
|
| 84 |
+
hosted unmodified. Its author published no checkpoint, so the ONNX is the
|
| 85 |
+
only form of that model that exists. CamStack's modification is the OpenVINO
|
| 86 |
+
conversion, which the script reproduces from it.
|
| 87 |
+
- `coreml/` and `tflite/` are Ultralytics exports of `best.pt` of
|
| 88 |
+
[Koushim/yolov8-license-plate-detection](https://huggingface.co/Koushim/yolov8-license-plate-detection)
|
| 89 |
+
(revision `9aaa5cd4`, SHA-256 `2d95861825bb4184404344c9cf809f40fd31dba785fe54e8ba5b9a3583789822`).
|
| 90 |
+
The script downloads and verifies both files.
|
| 91 |
+
|
| 92 |
+
Both model cards say MIT. The Ultralytics YOLOv8n base weights keep both
|
| 93 |
+
AGPL-3.0, as their own metadata states. Neither publishes its training data.
|
| 94 |
|
| 95 |
## Licences
|
| 96 |
|
source/build-camstack-models.py
CHANGED
|
@@ -1,265 +1,670 @@
|
|
| 1 |
#!/usr/bin/env python3
|
| 2 |
"""
|
| 3 |
-
|
| 4 |
-
|
| 5 |
-
The pipeline's Python pool currently does N predicts iterated internally
|
| 6 |
-
on a list-of-dicts (saves session setup, but each frame still costs ~9ms
|
| 7 |
-
warm). A CoreML model exported with a flexible batch dim accepts a single
|
| 8 |
-
stacked tensor of shape `(N, 3, H, W)` — one ANE dispatch per batch,
|
| 9 |
-
amortising every fixed cost.
|
| 10 |
-
|
| 11 |
-
This script produces (per model in MODELS):
|
| 12 |
-
exported-models/camstack-<id>.onnx — ONNX with dynamic batch axis
|
| 13 |
-
exported-models/camstack-<id>.mlpackage — CoreML with RangeDim batch (1..16)
|
| 14 |
-
exported-models/camstack-<id>_openvino_model/ — OpenVINO IR with dynamic batch
|
| 15 |
-
|
| 16 |
-
The dynamic axis is SAFE to export and must not be pinned here: the same
|
| 17 |
-
artifact is loaded on CPU, iGPU, NPU and ANE nodes, so an export cannot know
|
| 18 |
-
which target it will meet. The loader does — `_static_batch_required` /
|
| 19 |
-
`_pin_batch_for_device` in `packages/addon-pipeline/python/inference_pool.py`
|
| 20 |
-
reshape the graph to a static batch of 1 for the one target that refuses dynamic
|
| 21 |
-
dims (the Intel NPU) and hand it over untouched everywhere else (D202). Pinning
|
| 22 |
-
it here would instead delete the flexible axis the CoreML batched-ANE dispatch
|
| 23 |
-
path depends on.
|
| 24 |
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 28 |
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 32 |
"""
|
| 33 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 34 |
import os
|
| 35 |
import shutil
|
|
|
|
| 36 |
import sys
|
|
|
|
|
|
|
| 37 |
from pathlib import Path
|
|
|
|
| 38 |
|
| 39 |
-
# convert_lib.py is the converter Model Studio runs at runtime. In the CamStack
|
| 40 |
-
# repository it lives in packages/addon-model-studio/python/; the published copy
|
| 41 |
-
# of this script (the `source/` folder of the camstack/camstack-models mirror)
|
| 42 |
-
# ships it as a sibling file. A sibling wins, so the SAME file runs in both places.
|
| 43 |
_HERE = Path(__file__).resolve().parent
|
| 44 |
-
_CONVERT_LIB_DIRS = (
|
| 45 |
-
|
| 46 |
-
_HERE.parent / 'packages' / 'addon-model-studio' / 'python',
|
| 47 |
-
)
|
| 48 |
-
_CONVERT_LIB_DIR = next(
|
| 49 |
-
(d for d in _CONVERT_LIB_DIRS if (d / 'convert_lib.py').is_file()), None
|
| 50 |
-
)
|
| 51 |
if _CONVERT_LIB_DIR is None:
|
| 52 |
-
sys.exit(
|
| 53 |
-
'convert_lib.py not found next to this script or in '
|
| 54 |
-
'packages/addon-model-studio/python/'
|
| 55 |
-
)
|
| 56 |
sys.path.insert(0, str(_CONVERT_LIB_DIR))
|
| 57 |
-
from convert_lib import export_openvino, quantize_openvino_int8, validate_openvino_ir
|
| 58 |
-
|
| 59 |
-
# --- Config ------------------------------------------------------------
|
| 60 |
|
| 61 |
-
|
|
|
|
| 62 |
|
| 63 |
-
|
| 64 |
-
# (ultralytics-id, exported-name, image-size)
|
| 65 |
-
("yolov9t", "camstack-yolov9t", 640),
|
| 66 |
-
("yolov9s", "camstack-yolov9s", 640),
|
| 67 |
-
("yolo26n", "camstack-yolo26n", 640),
|
| 68 |
-
]
|
| 69 |
|
| 70 |
-
# Directory of calibration images (e.g. a COCO val subset) for INT8. INT8 is
|
| 71 |
-
# only produced when this dir exists + has images; otherwise it's skipped with
|
| 72 |
-
# a note. Override via CAMSTACK_CALIB_DIR.
|
| 73 |
-
CALIB_DIR = Path(os.environ.get("CAMSTACK_CALIB_DIR", "calibration-images"))
|
| 74 |
|
| 75 |
-
|
| 76 |
-
|
| 77 |
-
|
| 78 |
|
|
|
|
|
|
|
|
|
|
| 79 |
|
| 80 |
-
# --- Helpers -----------------------------------------------------------
|
| 81 |
|
|
|
|
|
|
|
|
|
|
| 82 |
|
| 83 |
-
|
| 84 |
-
|
| 85 |
-
return path.stat().st_size / (1024 * 1024)
|
| 86 |
-
return sum(f.stat().st_size for f in path.rglob("*") if f.is_file()) / (1024 * 1024)
|
| 87 |
|
| 88 |
|
| 89 |
-
|
| 90 |
-
|
| 91 |
-
|
| 92 |
|
|
|
|
|
|
|
|
|
|
| 93 |
|
| 94 |
-
# --- Step 1: YOLO -> ONNX (dynamic batch) ------------------------------
|
| 95 |
|
|
|
|
|
|
|
|
|
|
| 96 |
|
| 97 |
-
|
| 98 |
-
|
|
|
|
| 99 |
|
| 100 |
-
|
| 101 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 102 |
"""
|
| 103 |
-
|
| 104 |
-
|
| 105 |
-
|
| 106 |
-
|
| 107 |
-
|
| 108 |
-
|
| 109 |
-
|
| 110 |
-
imgsz=imgsz,
|
| 111 |
-
dynamic=True,
|
| 112 |
-
simplify=True,
|
| 113 |
-
opset=17,
|
| 114 |
)
|
| 115 |
-
if not onnx_src:
|
| 116 |
-
raise RuntimeError(f"ONNX export returned no path for {ultra_id}")
|
| 117 |
-
onnx_src = Path(onnx_src)
|
| 118 |
-
if onnx_src != out_path:
|
| 119 |
-
shutil.copy2(onnx_src, out_path)
|
| 120 |
-
print(f" ✓ {out_path.name} ({get_size_mb(out_path):.1f} MB)")
|
| 121 |
-
return out_path
|
| 122 |
|
| 123 |
|
| 124 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 125 |
|
| 126 |
|
| 127 |
-
def
|
| 128 |
-
"""
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 129 |
|
| 130 |
-
Ultralytics' CoreML exporter handles the torch.jit.trace + coremltools
|
| 131 |
-
pipeline internally. With `dynamic=True` + `batch=N`, it produces a
|
| 132 |
-
model that accepts shape `(B, 3, imgsz, imgsz)` with B ∈ [1..N].
|
| 133 |
-
`half=True` sets compute precision to FLOAT16 — required for ANE.
|
| 134 |
-
"""
|
| 135 |
-
from ultralytics import YOLO
|
| 136 |
|
| 137 |
-
|
| 138 |
-
|
| 139 |
-
|
| 140 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 141 |
)
|
| 142 |
-
|
| 143 |
-
|
| 144 |
-
|
| 145 |
-
|
| 146 |
-
|
| 147 |
-
|
| 148 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 149 |
)
|
| 150 |
-
|
| 151 |
-
|
| 152 |
-
src = Path(src)
|
| 153 |
-
if out_path.exists():
|
| 154 |
-
shutil.rmtree(out_path)
|
| 155 |
-
if src.is_dir():
|
| 156 |
-
shutil.copytree(src, out_path)
|
| 157 |
-
else:
|
| 158 |
-
out_path.mkdir(parents=True, exist_ok=True)
|
| 159 |
-
shutil.copy2(src, out_path / src.name)
|
| 160 |
-
print(f" ✓ {out_path.name} ({get_size_mb(out_path):.1f} MB)")
|
| 161 |
-
return out_path
|
| 162 |
-
|
| 163 |
-
|
| 164 |
-
# --- Step 3: ONNX -> OpenVINO (dynamic batch IR) -----------------------
|
| 165 |
-
# export_openvino, quantize_openvino_int8, validate_openvino_ir are imported
|
| 166 |
-
# from convert_lib.py (shared with the runtime; located above).
|
| 167 |
-
|
| 168 |
-
# --- Step 4: Verification ---------------------------------------------
|
| 169 |
-
|
| 170 |
-
|
| 171 |
-
def verify_coreml_batch(mlpackage_path: Path, imgsz: int) -> None:
|
| 172 |
-
"""Predict with N=1 then N=4 to confirm batch flexibility works."""
|
| 173 |
-
import coremltools as ct
|
| 174 |
-
import numpy as np
|
| 175 |
-
|
| 176 |
-
print(f" → loading {mlpackage_path.name} for verification")
|
| 177 |
-
model = ct.models.MLModel(str(mlpackage_path))
|
| 178 |
-
# Find input name
|
| 179 |
-
spec = model.get_spec()
|
| 180 |
-
input_name = spec.description.input[0].name
|
| 181 |
-
|
| 182 |
-
for batch in (1, 4):
|
| 183 |
-
x = np.random.rand(batch, 3, imgsz, imgsz).astype(np.float32)
|
| 184 |
-
out = model.predict({input_name: x})
|
| 185 |
-
out_keys = list(out.keys())
|
| 186 |
-
out_shape = out[out_keys[0]].shape if hasattr(out[out_keys[0]], "shape") else "?"
|
| 187 |
-
print(f" ✓ batch={batch} → output[{out_keys[0]}].shape={out_shape}")
|
| 188 |
-
|
| 189 |
-
|
| 190 |
-
# --- Main --------------------------------------------------------------
|
| 191 |
-
|
| 192 |
-
|
| 193 |
-
def main() -> None:
|
| 194 |
-
output = ensure_dir(OUTPUT_DIR)
|
| 195 |
-
print(f"Output: {output.resolve()}")
|
| 196 |
-
|
| 197 |
-
for ultra_id, name, imgsz in MODELS:
|
| 198 |
-
print()
|
| 199 |
-
print("=" * 70)
|
| 200 |
-
print(f"Building {name} (source: {ultra_id}.pt, imgsz={imgsz}×{imgsz})")
|
| 201 |
-
print("=" * 70)
|
| 202 |
-
|
| 203 |
-
onnx_path = output / f"{name}.onnx"
|
| 204 |
-
coreml_path = output / f"{name}.mlpackage"
|
| 205 |
-
|
| 206 |
-
try:
|
| 207 |
-
export_onnx(ultra_id, onnx_path, imgsz)
|
| 208 |
-
except Exception as e:
|
| 209 |
-
print(f" ✗ ONNX export failed: {e}")
|
| 210 |
-
continue
|
| 211 |
|
| 212 |
-
|
| 213 |
-
|
| 214 |
-
|
| 215 |
-
|
| 216 |
-
|
| 217 |
-
|
| 218 |
-
|
| 219 |
-
|
| 220 |
-
|
| 221 |
-
|
| 222 |
-
|
| 223 |
-
|
| 224 |
-
|
| 225 |
-
|
| 226 |
-
|
| 227 |
-
|
| 228 |
-
|
| 229 |
-
|
| 230 |
-
|
| 231 |
-
|
| 232 |
-
|
| 233 |
-
|
| 234 |
-
|
| 235 |
-
|
| 236 |
-
|
| 237 |
-
|
| 238 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 239 |
else:
|
| 240 |
-
|
| 241 |
-
|
| 242 |
-
|
| 243 |
-
|
| 244 |
-
|
| 245 |
-
|
| 246 |
-
|
| 247 |
-
|
| 248 |
-
|
| 249 |
-
|
| 250 |
-
|
| 251 |
-
|
| 252 |
-
|
| 253 |
-
|
| 254 |
-
|
| 255 |
-
|
| 256 |
-
|
| 257 |
-
|
| 258 |
-
|
| 259 |
-
|
| 260 |
-
|
| 261 |
-
|
| 262 |
-
|
| 263 |
-
|
| 264 |
-
|
| 265 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
#!/usr/bin/env python3
|
| 2 |
"""
|
| 3 |
+
Rebuild every Ultralytics-derived (AGPL-3.0) model the CamStack mirror hosts.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
|
| 5 |
+
This is the Corresponding Source for the AGPL-3.0 weights of
|
| 6 |
+
https://huggingface.co/camstack/camstack-models in
|
| 7 |
+
|
| 8 |
+
objectDetection/yolo26 yolo26{n,s,m,l,x} @640, yolo26{n,s,m,l} @320 and @256
|
| 9 |
+
objectDetection/yolov9 yolov9{t,s,m,c} @320 and @640
|
| 10 |
+
segmentation/yolo26-seg yolo26{n,s,m}-seg @640
|
| 11 |
+
plateDetection/yolov8-plate yolov8n-plate (third-party fine-tune, see below)
|
| 12 |
|
| 13 |
+
Every hosted file is one output of ONE recipe in `RECIPES` below, and the recipe
|
| 14 |
+
holds exactly the settings that file records about itself: the Ultralytics
|
| 15 |
+
`export()` arguments (stored in each ONNX / CoreML file's metadata), the OpenVINO
|
| 16 |
+
weight compression and the NNCF calibration subset size (stored in each IR's
|
| 17 |
+
`rt_info`). `--check-hosted` proves the mapping is complete against the live
|
| 18 |
+
mirror: every hosted weight file must be an output of some recipe.
|
| 19 |
+
|
| 20 |
+
Usage:
|
| 21 |
+
python build-camstack-models.py --list # every recipe
|
| 22 |
+
python build-camstack-models.py --check-hosted # recipes == mirror?
|
| 23 |
+
python build-camstack-models.py --variant camstack-yolo26n-320
|
| 24 |
+
python build-camstack-models.py --family yolov9 # one mirror folder
|
| 25 |
+
python build-camstack-models.py --all # the whole mirror
|
| 26 |
+
python build-camstack-models.py --all --formats onnx,openvino --dry-run
|
| 27 |
+
|
| 28 |
+
# Something not in the mirror (same pipeline, your own settings):
|
| 29 |
+
python build-camstack-models.py --model yolo26n --imgsz 416 \
|
| 30 |
+
--openvino fp16,int8 --calib-samples 128
|
| 31 |
+
|
| 32 |
+
Outputs land in `--out` (default ./exported-models) under the MIRROR's paths,
|
| 33 |
+
e.g. exported-models/objectDetection/yolo26/openvino/camstack-yolo26n-320-int8.xml.
|
| 34 |
+
|
| 35 |
+
Formats:
|
| 36 |
+
onnx Ultralytics export(format="onnx") from the upstream .pt
|
| 37 |
+
coreml Ultralytics export(format="coreml") from the upstream .pt (macOS only)
|
| 38 |
+
openvino convert_lib.export_openvino: ov.convert_model(ONNX) + ov.save_model
|
| 39 |
+
(compress_to_fp16 per precision) — never the Ultralytics OV exporter
|
| 40 |
+
int8 convert_lib.quantize_openvino_int8: NNCF post-training quantisation
|
| 41 |
+
of the ONNX, calibrated on the first N images (sorted by file name)
|
| 42 |
+
of --calib-dir (default $CAMSTACK_CALIB_DIR or ./calibration-images)
|
| 43 |
+
tflite Ultralytics export(format="tflite") (needs tensorflow + onnx2tf)
|
| 44 |
+
|
| 45 |
+
INT8 calibration: the hosted INT8 IRs record subset_size 128 (YOLO26) and 300
|
| 46 |
+
(YOLOv9), drawn from COCO; an IR does not record WHICH images. The documented
|
| 47 |
+
set is COCO val2017: unpack http://images.cocodataset.org/zips/val2017.zip and
|
| 48 |
+
point --calib-dir at it — the recipe's N is taken in file-name order. Images are letterboxed to
|
| 49 |
+
the recipe's input size with grey (114) padding and scaled to [0, 1], exactly
|
| 50 |
+
as the CamStack runtime feeds the model. A different calibration set yields an
|
| 51 |
+
equivalent but not bit-identical IR (NNCF picks ranges from the data).
|
| 52 |
+
|
| 53 |
+
Toolchain: each recipe names the Ultralytics / torch / coremltools / OpenVINO
|
| 54 |
+
versions its hosted files record. Other versions produce the same model with a
|
| 55 |
+
possibly different graph layout; the script warns when they differ.
|
| 56 |
+
|
| 57 |
+
Environment: `pip install -r requirements.txt ultralytics` (requirements.txt is
|
| 58 |
+
convert_lib's). convert_lib.py is found next to this script (the mirror's
|
| 59 |
+
`source/` folder) or in packages/addon-model-studio/python/ (the repository).
|
| 60 |
"""
|
| 61 |
|
| 62 |
+
from __future__ import annotations
|
| 63 |
+
|
| 64 |
+
import argparse
|
| 65 |
+
import hashlib
|
| 66 |
+
import json
|
| 67 |
import os
|
| 68 |
import shutil
|
| 69 |
+
import subprocess
|
| 70 |
import sys
|
| 71 |
+
import urllib.request
|
| 72 |
+
from dataclasses import dataclass, field
|
| 73 |
from pathlib import Path
|
| 74 |
+
from typing import Callable, Optional
|
| 75 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 76 |
_HERE = Path(__file__).resolve().parent
|
| 77 |
+
_CONVERT_LIB_DIRS = (_HERE, _HERE.parent / 'packages' / 'addon-model-studio' / 'python')
|
| 78 |
+
_CONVERT_LIB_DIR = next((d for d in _CONVERT_LIB_DIRS if (d / 'convert_lib.py').is_file()), None)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 79 |
if _CONVERT_LIB_DIR is None:
|
| 80 |
+
sys.exit('convert_lib.py not found next to this script or in packages/addon-model-studio/python/')
|
|
|
|
|
|
|
|
|
|
| 81 |
sys.path.insert(0, str(_CONVERT_LIB_DIR))
|
|
|
|
|
|
|
|
|
|
| 82 |
|
| 83 |
+
HF_REPO = 'camstack/camstack-models'
|
| 84 |
+
DEFAULT_CALIB_DIR = Path(os.environ.get('CAMSTACK_CALIB_DIR', 'calibration-images'))
|
| 85 |
|
| 86 |
+
# --- Recipe model ------------------------------------------------------------
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 87 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 88 |
|
| 89 |
+
@dataclass(frozen=True)
|
| 90 |
+
class Download:
|
| 91 |
+
"""A third-party file fetched by URL and verified by SHA-256."""
|
| 92 |
|
| 93 |
+
url: str
|
| 94 |
+
sha256: str
|
| 95 |
+
filename: str
|
| 96 |
|
|
|
|
| 97 |
|
| 98 |
+
@dataclass(frozen=True)
|
| 99 |
+
class OnnxExport:
|
| 100 |
+
"""Ultralytics export(format="onnx") arguments, as the hosted ONNX records them."""
|
| 101 |
|
| 102 |
+
dynamic: bool
|
| 103 |
+
opset: int
|
|
|
|
|
|
|
| 104 |
|
| 105 |
|
| 106 |
+
@dataclass(frozen=True)
|
| 107 |
+
class CoremlExport:
|
| 108 |
+
"""Ultralytics export(format="coreml") arguments, as the hosted package records them."""
|
| 109 |
|
| 110 |
+
dynamic: bool
|
| 111 |
+
batch: int
|
| 112 |
+
half: bool
|
| 113 |
|
|
|
|
| 114 |
|
| 115 |
+
@dataclass(frozen=True)
|
| 116 |
+
class OpenvinoOutput:
|
| 117 |
+
"""One hosted IR: file suffix after the model name, and how it is produced.
|
| 118 |
|
| 119 |
+
`precision` is `fp32` (compress_to_fp16=False), `fp16` (compress_to_fp16=True)
|
| 120 |
+
or `int8` (NNCF on the ONNX, `calib_samples` images).
|
| 121 |
+
"""
|
| 122 |
|
| 123 |
+
suffix: str
|
| 124 |
+
precision: str
|
| 125 |
+
|
| 126 |
+
|
| 127 |
+
@dataclass(frozen=True)
|
| 128 |
+
class Toolchain:
|
| 129 |
+
ultralytics: str
|
| 130 |
+
torch: str
|
| 131 |
+
coremltools: str
|
| 132 |
+
openvino: str
|
| 133 |
+
|
| 134 |
+
|
| 135 |
+
@dataclass(frozen=True)
|
| 136 |
+
class Recipe:
|
| 137 |
+
name: str # hosted file stem, e.g. camstack-yolo26n-320
|
| 138 |
+
folder: str # mirror folder, e.g. objectDetection/yolo26
|
| 139 |
+
imgsz: int
|
| 140 |
+
toolchain: Toolchain
|
| 141 |
+
# The PyTorch checkpoint: an Ultralytics asset id (`yolo26n` -> yolo26n.pt,
|
| 142 |
+
# auto-downloaded by Ultralytics) or a third-party Download.
|
| 143 |
+
checkpoint: str | Download
|
| 144 |
+
onnx: Optional[OnnxExport] # None: the ONNX is `onnx_verbatim`, not exported
|
| 145 |
+
coreml: Optional[CoremlExport]
|
| 146 |
+
openvino: tuple[OpenvinoOutput, ...] = ()
|
| 147 |
+
calib_samples: int = 0
|
| 148 |
+
tflite: bool = False
|
| 149 |
+
onnx_verbatim: Optional[Download] = None # a third-party ONNX hosted unmodified
|
| 150 |
+
note: str = ''
|
| 151 |
+
|
| 152 |
+
def outputs(self) -> list[str]:
|
| 153 |
+
"""Mirror paths this recipe produces (an .mlpackage is a directory)."""
|
| 154 |
+
out: list[str] = []
|
| 155 |
+
if self.onnx is not None or self.onnx_verbatim is not None:
|
| 156 |
+
out.append(f'{self.folder}/onnx/{self.name}.onnx')
|
| 157 |
+
if self.coreml is not None:
|
| 158 |
+
out.append(f'{self.folder}/coreml/{self.name}.mlpackage/')
|
| 159 |
+
for ov in self.openvino:
|
| 160 |
+
stem = f'{self.folder}/openvino/{self.name}{ov.suffix}'
|
| 161 |
+
out.extend([f'{stem}.xml', f'{stem}.bin'])
|
| 162 |
+
if self.tflite:
|
| 163 |
+
out.append(f'{self.folder}/tflite/{self.name}_float32.tflite')
|
| 164 |
+
return out
|
| 165 |
+
|
| 166 |
+
|
| 167 |
+
# --- The hosted set ------------------------------------------------------------
|
| 168 |
+
# Toolchains as recorded by the hosted files themselves (ONNX/CoreML metadata
|
| 169 |
+
# `version`, CoreML `com.github.apple.coremltools.*` and `torch==`, IR rt_info
|
| 170 |
+
# `Runtime_version`).
|
| 171 |
+
TC_2604 = Toolchain('8.4.24', '2.10.0', '9.0', '2026.2.1')
|
| 172 |
+
TC_2607_REDUCED = Toolchain('8.4.92', 'onnx 2.13.0, coreml 2.5.1', '8.3.0', '2026.2.1')
|
| 173 |
+
TC_2607_YOLOV9 = Toolchain('8.4.98', '2.13.0', '9.0', '2026.2.1')
|
| 174 |
+
TC_PLATE = Toolchain('8.4.24 (CoreML, TFLite)', '2.10.0', '9.0', '2026.0.0')
|
| 175 |
+
|
| 176 |
+
FP16_IR = OpenvinoOutput('', 'fp16')
|
| 177 |
+
|
| 178 |
+
|
| 179 |
+
def _yolo26_full(tier: str) -> Recipe:
|
| 180 |
+
"""yolo26<tier> @640: static ONNX, static fp32-compute CoreML, IR fp16 twice + INT8.
|
| 181 |
+
|
| 182 |
+
The unsuffixed IR and `-fp16` IR are the same compressed-to-FP16 weights;
|
| 183 |
+
the catalog addresses both names.
|
| 184 |
"""
|
| 185 |
+
return Recipe(
|
| 186 |
+
name=f'camstack-yolo26{tier}', folder='objectDetection/yolo26', imgsz=640,
|
| 187 |
+
toolchain=TC_2604, checkpoint=f'yolo26{tier}',
|
| 188 |
+
onnx=OnnxExport(dynamic=False, opset=17),
|
| 189 |
+
coreml=CoremlExport(dynamic=False, batch=1, half=False),
|
| 190 |
+
openvino=(FP16_IR, OpenvinoOutput('-fp16', 'fp16'), OpenvinoOutput('-int8', 'int8')),
|
| 191 |
+
calib_samples=128,
|
|
|
|
|
|
|
|
|
|
|
|
|
| 192 |
)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 193 |
|
| 194 |
|
| 195 |
+
def _yolo26_reduced(tier: str, res: int) -> Recipe:
|
| 196 |
+
"""yolo26<tier>-<res>: dynamic-axes ONNX, batch-flexible (1..16) FP16 CoreML, IR fp32 + INT8."""
|
| 197 |
+
return Recipe(
|
| 198 |
+
name=f'camstack-yolo26{tier}-{res}', folder='objectDetection/yolo26', imgsz=res,
|
| 199 |
+
toolchain=TC_2607_REDUCED, checkpoint=f'yolo26{tier}',
|
| 200 |
+
onnx=OnnxExport(dynamic=True, opset=17),
|
| 201 |
+
coreml=CoremlExport(dynamic=True, batch=16, half=True),
|
| 202 |
+
openvino=(OpenvinoOutput('', 'fp32'), OpenvinoOutput('-int8', 'int8')),
|
| 203 |
+
calib_samples=128,
|
| 204 |
+
)
|
| 205 |
|
| 206 |
|
| 207 |
+
def _yolov9(tier: str, res: int) -> Recipe:
|
| 208 |
+
"""yolov9<tier>-<res>: static opset-13 ONNX, static CoreML, IR fp16 + INT8."""
|
| 209 |
+
return Recipe(
|
| 210 |
+
name=f'camstack-yolov9{tier}-{res}', folder='objectDetection/yolov9', imgsz=res,
|
| 211 |
+
toolchain=TC_2607_YOLOV9, checkpoint=f'yolov9{tier}',
|
| 212 |
+
onnx=OnnxExport(dynamic=False, opset=13),
|
| 213 |
+
coreml=CoremlExport(dynamic=False, batch=1, half=False),
|
| 214 |
+
openvino=(OpenvinoOutput('-fp16', 'fp16'), OpenvinoOutput('-int8', 'int8')),
|
| 215 |
+
calib_samples=300,
|
| 216 |
+
)
|
| 217 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 218 |
|
| 219 |
+
def _yolo26_seg(tier: str) -> Recipe:
|
| 220 |
+
return Recipe(
|
| 221 |
+
name=f'camstack-yolo26{tier}-seg', folder='segmentation/yolo26-seg', imgsz=640,
|
| 222 |
+
toolchain=TC_2604, checkpoint=f'yolo26{tier}-seg',
|
| 223 |
+
onnx=OnnxExport(dynamic=False, opset=17),
|
| 224 |
+
coreml=CoremlExport(dynamic=False, batch=1, half=False),
|
| 225 |
+
openvino=(FP16_IR,),
|
| 226 |
+
)
|
| 227 |
+
|
| 228 |
+
|
| 229 |
+
# yolov8n-plate is TWO third-party YOLOv8n licence-plate fine-tunes, neither
|
| 230 |
+
# trained by CamStack (both declare MIT on their model cards; the Ultralytics
|
| 231 |
+
# base weights make them AGPL-3.0, and their own metadata says so):
|
| 232 |
+
# - the ONNX (and the OpenVINO IR converted from it) is ml-debi's `best.onnx`,
|
| 233 |
+
# hosted byte-identical. Its author published no .pt; the ONNX is the only
|
| 234 |
+
# form that exists. CamStack's modification is the IR conversion below.
|
| 235 |
+
# - the CoreML package and the TFLite file are Ultralytics exports of
|
| 236 |
+
# Koushim's `best.pt`, saved as `yolov8n-plate.pt` (the file stem is what
|
| 237 |
+
# the export metadata calls the model). The .pt is public.
|
| 238 |
+
_PLATE_ONNX = Download(
|
| 239 |
+
url='https://huggingface.co/ml-debi/yolov8-license-plate-detection/resolve/'
|
| 240 |
+
'8062e8e86734e272c60fa9a819433327ebdb66d7/best.onnx',
|
| 241 |
+
sha256='85d236280a1301ad98907947d284951dd2b20c23a6786ff50f7e6a8ec515bd50',
|
| 242 |
+
filename='camstack-yolov8n-plate.onnx',
|
| 243 |
+
)
|
| 244 |
+
_PLATE_PT = Download(
|
| 245 |
+
url='https://huggingface.co/Koushim/yolov8-license-plate-detection/resolve/'
|
| 246 |
+
'9aaa5cd490abe0c165882ba87f4f62658ab54d01/best.pt',
|
| 247 |
+
sha256='2d95861825bb4184404344c9cf809f40fd31dba785fe54e8ba5b9a3583789822',
|
| 248 |
+
filename='yolov8n-plate.pt',
|
| 249 |
+
)
|
| 250 |
+
|
| 251 |
+
PLATE = Recipe(
|
| 252 |
+
name='camstack-yolov8n-plate', folder='plateDetection/yolov8-plate', imgsz=640,
|
| 253 |
+
toolchain=TC_PLATE, checkpoint=_PLATE_PT,
|
| 254 |
+
onnx=None, onnx_verbatim=_PLATE_ONNX,
|
| 255 |
+
coreml=CoremlExport(dynamic=False, batch=1, half=False),
|
| 256 |
+
openvino=(FP16_IR,), tflite=True,
|
| 257 |
+
note='ONNX + IR: ml-debi/yolov8-license-plate-detection; CoreML + TFLite: '
|
| 258 |
+
'Koushim/yolov8-license-plate-detection (different fine-tunes)',
|
| 259 |
+
)
|
| 260 |
+
|
| 261 |
+
RECIPES: tuple[Recipe, ...] = (
|
| 262 |
+
*(_yolo26_full(t) for t in 'nsmlx'),
|
| 263 |
+
*(_yolo26_reduced(t, r) for t in 'nsml' for r in (320, 256)),
|
| 264 |
+
*(_yolov9(t, r) for t in ('t', 's', 'm', 'c') for r in (320, 640)),
|
| 265 |
+
*(_yolo26_seg(t) for t in 'nsm'),
|
| 266 |
+
PLATE,
|
| 267 |
+
)
|
| 268 |
+
|
| 269 |
+
FAMILIES = {
|
| 270 |
+
'yolo26': 'objectDetection/yolo26',
|
| 271 |
+
'yolov9': 'objectDetection/yolov9',
|
| 272 |
+
'yolo26-seg': 'segmentation/yolo26-seg',
|
| 273 |
+
'yolov8-plate': 'plateDetection/yolov8-plate',
|
| 274 |
+
}
|
| 275 |
+
ALL_FORMATS = ('onnx', 'coreml', 'openvino', 'tflite')
|
| 276 |
+
|
| 277 |
+
# --- Helpers ----------------------------------------------------------------
|
| 278 |
+
|
| 279 |
+
|
| 280 |
+
def size_mb(path: Path) -> float:
|
| 281 |
+
if path.is_file():
|
| 282 |
+
return path.stat().st_size / 1_048_576
|
| 283 |
+
return sum(f.stat().st_size for f in path.rglob('*') if f.is_file()) / 1_048_576
|
| 284 |
+
|
| 285 |
+
|
| 286 |
+
def sha256_of(path: Path) -> str:
|
| 287 |
+
h = hashlib.sha256()
|
| 288 |
+
with path.open('rb') as fh:
|
| 289 |
+
for chunk in iter(lambda: fh.read(1 << 20), b''):
|
| 290 |
+
h.update(chunk)
|
| 291 |
+
return h.hexdigest()
|
| 292 |
+
|
| 293 |
+
|
| 294 |
+
def fetch(dl: Download, cache: Path) -> Path:
|
| 295 |
+
"""Download a third-party file once and refuse it if the SHA-256 differs."""
|
| 296 |
+
cache.mkdir(parents=True, exist_ok=True)
|
| 297 |
+
dest = cache / dl.filename
|
| 298 |
+
if not dest.is_file():
|
| 299 |
+
print(f' → downloading {dl.url}')
|
| 300 |
+
urllib.request.urlretrieve(dl.url, dest)
|
| 301 |
+
got = sha256_of(dest)
|
| 302 |
+
if got != dl.sha256:
|
| 303 |
+
raise RuntimeError(f'{dest} has sha256 {got}, expected {dl.sha256}')
|
| 304 |
+
return dest
|
| 305 |
+
|
| 306 |
+
|
| 307 |
+
def warn_toolchain(recipe: Recipe) -> None:
|
| 308 |
+
try:
|
| 309 |
+
import ultralytics
|
| 310 |
+
except ImportError:
|
| 311 |
+
return
|
| 312 |
+
want = recipe.toolchain.ultralytics.split(' ')[0]
|
| 313 |
+
if ultralytics.__version__ != want:
|
| 314 |
+
print(f' ! the hosted {recipe.name} files were made with ultralytics {want}; '
|
| 315 |
+
f'this is {ultralytics.__version__} (same weights, the graph may differ)')
|
| 316 |
+
|
| 317 |
+
|
| 318 |
+
def place(src: Path, dest: Path) -> Path:
|
| 319 |
+
dest.parent.mkdir(parents=True, exist_ok=True)
|
| 320 |
+
if dest.exists():
|
| 321 |
+
shutil.rmtree(dest) if dest.is_dir() else dest.unlink()
|
| 322 |
+
(shutil.copytree if src.is_dir() else shutil.copy2)(src, dest)
|
| 323 |
+
print(f' ✓ {dest} ({size_mb(dest):.1f} MB)')
|
| 324 |
+
return dest
|
| 325 |
+
|
| 326 |
+
|
| 327 |
+
def checkpoint_path(recipe: Recipe, cache: Path) -> str:
|
| 328 |
+
if isinstance(recipe.checkpoint, Download):
|
| 329 |
+
return str(fetch(recipe.checkpoint, cache))
|
| 330 |
+
return f'{recipe.checkpoint}.pt' # Ultralytics downloads its own assets
|
| 331 |
+
|
| 332 |
+
|
| 333 |
+
# --- Child processes ----------------------------------------------------------
|
| 334 |
+
# Every backend runs in its OWN interpreter. On macOS two of them in one process
|
| 335 |
+
# abort it: an ONNX export followed by a CoreML export ("libc++abi: mutex lock
|
| 336 |
+
# failed"), and OpenVINO beside torch / onnx / tensorflow (each initialises its
|
| 337 |
+
# own protobuf). A child per step keeps a crash to that step, reported as a
|
| 338 |
+
# failure, and the parent imports none of them.
|
| 339 |
+
|
| 340 |
+
_CHILD_MARKER = '@@CAMSTACK_CHILD@@'
|
| 341 |
+
|
| 342 |
+
_ULTRALYTICS_CHILD = f"""
|
| 343 |
+
import json, sys
|
| 344 |
+
from ultralytics import YOLO
|
| 345 |
+
args = json.loads(sys.argv[1])
|
| 346 |
+
path = YOLO(args.pop('checkpoint')).export(**args)
|
| 347 |
+
print('\\n{_CHILD_MARKER}' + str(path), flush=True)
|
| 348 |
+
"""
|
| 349 |
+
|
| 350 |
+
# Only openvino (and, for INT8, nncf + cv2) is imported here: no torch, no onnx,
|
| 351 |
+
# no tensorflow. The conversion is convert_lib's, the same code Model Studio runs.
|
| 352 |
+
_OPENVINO_CHILD = f"""
|
| 353 |
+
import json, sys
|
| 354 |
+
args = json.loads(sys.argv[1])
|
| 355 |
+
sys.path.insert(0, args['convert_lib_dir'])
|
| 356 |
+
from convert_lib import export_openvino, quantize_openvino_int8, validate_openvino_ir
|
| 357 |
+
if args['precision'] == 'int8':
|
| 358 |
+
quantize_openvino_int8(args['onnx'], args['xml'], args['calib_dir'], args['imgsz'],
|
| 359 |
+
max_samples=args['calib_samples'])
|
| 360 |
+
else:
|
| 361 |
+
export_openvino(args['onnx'], args['xml'], fp16=args['precision'] == 'fp16')
|
| 362 |
+
validate_openvino_ir(args['xml'])
|
| 363 |
+
print('\\n{_CHILD_MARKER}' + args['xml'], flush=True)
|
| 364 |
+
"""
|
| 365 |
+
|
| 366 |
+
_COREML_CHECK_CHILD = f"""
|
| 367 |
+
import json, sys
|
| 368 |
+
import coremltools as ct
|
| 369 |
+
import numpy as np
|
| 370 |
+
args = json.loads(sys.argv[1])
|
| 371 |
+
model = ct.models.MLModel(args['package'])
|
| 372 |
+
name = model.get_spec().description.input[0].name
|
| 373 |
+
for batch in (1, 4):
|
| 374 |
+
out = model.predict({{name: np.random.rand(batch, 3, args['imgsz'], args['imgsz']).astype(np.float32)}})
|
| 375 |
+
print(f' batch={{batch}} -> {{getattr(next(iter(out.values())), "shape", "?")}}')
|
| 376 |
+
print('\\n{_CHILD_MARKER}ok', flush=True)
|
| 377 |
+
"""
|
| 378 |
+
|
| 379 |
+
|
| 380 |
+
def run_child(code: str, payload: dict[str, object], what: str) -> str:
|
| 381 |
+
"""Run one backend step in a fresh interpreter; return what it reported.
|
| 382 |
+
|
| 383 |
+
The step counts as done only when the child printed its marker line; a
|
| 384 |
+
child that crashes (a signal, an abort) before that is a failed step and is
|
| 385 |
+
never retried.
|
| 386 |
+
"""
|
| 387 |
+
proc = subprocess.run(
|
| 388 |
+
[sys.executable, '-c', code, json.dumps(payload)],
|
| 389 |
+
stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True,
|
| 390 |
)
|
| 391 |
+
marked = [ln[len(_CHILD_MARKER):] for ln in proc.stdout.splitlines() if ln.startswith(_CHILD_MARKER)]
|
| 392 |
+
if not marked:
|
| 393 |
+
sys.stdout.write('\n'.join(proc.stdout.splitlines()[-25:]) + '\n')
|
| 394 |
+
raise RuntimeError(f'{what} failed (child exit {proc.returncode})')
|
| 395 |
+
if proc.returncode != 0:
|
| 396 |
+
print(f' ! {what}: the child exited {proc.returncode} after finishing')
|
| 397 |
+
return marked[-1]
|
| 398 |
+
|
| 399 |
+
|
| 400 |
+
def ultralytics_export(checkpoint: str, **kwargs: object) -> Path:
|
| 401 |
+
path = Path(run_child(_ULTRALYTICS_CHILD, {'checkpoint': checkpoint, **kwargs},
|
| 402 |
+
f'ultralytics export {kwargs} of {checkpoint}'))
|
| 403 |
+
if not path.exists():
|
| 404 |
+
raise RuntimeError(f'ultralytics export reported {path}, which does not exist')
|
| 405 |
+
return path
|
| 406 |
+
|
| 407 |
+
|
| 408 |
+
# --- Format steps -------------------------------------------------------------
|
| 409 |
+
|
| 410 |
+
|
| 411 |
+
def build_onnx(recipe: Recipe, out: Path, cache: Path) -> Path:
|
| 412 |
+
dest = out / recipe.folder / 'onnx' / f'{recipe.name}.onnx'
|
| 413 |
+
if recipe.onnx_verbatim is not None:
|
| 414 |
+
print(' → ONNX: third-party file, hosted unmodified')
|
| 415 |
+
return place(fetch(recipe.onnx_verbatim, cache), dest)
|
| 416 |
+
assert recipe.onnx is not None
|
| 417 |
+
print(f' → ONNX (imgsz={recipe.imgsz}, dynamic={recipe.onnx.dynamic}, opset={recipe.onnx.opset})')
|
| 418 |
+
src = ultralytics_export(
|
| 419 |
+
checkpoint_path(recipe, cache), format='onnx', imgsz=recipe.imgsz,
|
| 420 |
+
dynamic=recipe.onnx.dynamic, simplify=True, opset=recipe.onnx.opset,
|
| 421 |
+
nms=False, batch=1, half=False,
|
| 422 |
)
|
| 423 |
+
return place(src, dest)
|
| 424 |
+
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 425 |
|
| 426 |
+
def build_coreml(recipe: Recipe, out: Path, cache: Path, verify: bool) -> Path:
|
| 427 |
+
assert recipe.coreml is not None
|
| 428 |
+
c = recipe.coreml
|
| 429 |
+
print(f' → CoreML (imgsz={recipe.imgsz}, dynamic={c.dynamic}, batch={c.batch}, half={c.half})')
|
| 430 |
+
src = ultralytics_export(
|
| 431 |
+
checkpoint_path(recipe, cache), format='coreml', imgsz=recipe.imgsz,
|
| 432 |
+
dynamic=c.dynamic, batch=c.batch, half=c.half, nms=False,
|
| 433 |
+
)
|
| 434 |
+
dest = place(src, out / recipe.folder / 'coreml' / f'{recipe.name}.mlpackage')
|
| 435 |
+
if verify and c.dynamic:
|
| 436 |
+
verify_coreml_batch(dest, recipe.imgsz)
|
| 437 |
+
return dest
|
| 438 |
+
|
| 439 |
+
|
| 440 |
+
def build_openvino(recipe: Recipe, onnx_path: Path, out: Path, calib_dir: Path,
|
| 441 |
+
calib_samples: int) -> list[str]:
|
| 442 |
+
"""Every IR of the recipe, each in its own child. Returns the outputs it skipped."""
|
| 443 |
+
skipped: list[str] = []
|
| 444 |
+
for ov in recipe.openvino:
|
| 445 |
+
xml = out / recipe.folder / 'openvino' / f'{recipe.name}{ov.suffix}.xml'
|
| 446 |
+
if ov.precision == 'int8' and not (calib_dir.is_dir() and any(calib_dir.iterdir())):
|
| 447 |
+
print(f' ⊘ {xml.name}: no calibration images in {calib_dir} (--calib-dir)')
|
| 448 |
+
skipped.append(str(xml))
|
| 449 |
+
continue
|
| 450 |
+
detail = f'NNCF, {calib_samples} images from {calib_dir}' if ov.precision == 'int8' else 'ov.convert_model'
|
| 451 |
+
print(f' → OpenVINO {ov.precision.upper()} {xml.name} ({detail})')
|
| 452 |
+
xml.parent.mkdir(parents=True, exist_ok=True)
|
| 453 |
+
run_child(_OPENVINO_CHILD, {
|
| 454 |
+
'convert_lib_dir': str(_CONVERT_LIB_DIR), 'onnx': str(onnx_path.resolve()),
|
| 455 |
+
'xml': str(xml.resolve()), 'precision': ov.precision, 'imgsz': recipe.imgsz,
|
| 456 |
+
'calib_dir': str(calib_dir.resolve()), 'calib_samples': calib_samples,
|
| 457 |
+
}, f'OpenVINO {ov.precision} {xml.name}')
|
| 458 |
+
print(f' ✓ {xml} ({size_mb(xml.with_suffix(".bin")):.1f} MB)')
|
| 459 |
+
return skipped
|
| 460 |
+
|
| 461 |
+
|
| 462 |
+
def build_tflite(recipe: Recipe, out: Path, cache: Path) -> Path:
|
| 463 |
+
print(f' → TFLite float32 (imgsz={recipe.imgsz})')
|
| 464 |
+
saved = ultralytics_export(checkpoint_path(recipe, cache), format='tflite', imgsz=recipe.imgsz)
|
| 465 |
+
stem = Path(checkpoint_path(recipe, cache)).stem
|
| 466 |
+
src = next(saved.rglob(f'{stem}_float32.tflite'), None) if saved.is_dir() else saved
|
| 467 |
+
if src is None:
|
| 468 |
+
raise RuntimeError(f'no {stem}_float32.tflite under {saved}')
|
| 469 |
+
return place(src, out / recipe.folder / 'tflite' / f'{recipe.name}_float32.tflite')
|
| 470 |
+
|
| 471 |
+
|
| 472 |
+
def verify_coreml_batch(mlpackage: Path, imgsz: int) -> None:
|
| 473 |
+
"""Predict with N=1 then N=4 on a flexible-batch package, in a throwaway child.
|
| 474 |
+
|
| 475 |
+
Opt-in (--verify-coreml): CoreML's runtime has aborted the whole process on
|
| 476 |
+
one of these packages (Espresso gather_nd shape computation) on a Mac whose
|
| 477 |
+
toolchain differs from the recorded one. A crash here is a failed CHECK, not
|
| 478 |
+
a failed build — the package on disk is the export's output either way.
|
| 479 |
+
"""
|
| 480 |
+
try:
|
| 481 |
+
run_child(_COREML_CHECK_CHILD, {'package': str(mlpackage.resolve()), 'imgsz': imgsz},
|
| 482 |
+
f'CoreML batch check of {mlpackage.name}')
|
| 483 |
+
print(' ✓ flexible batch predicts at N=1 and N=4')
|
| 484 |
+
except RuntimeError as exc:
|
| 485 |
+
print(f' ! {exc}; the package is kept, check it on the target device')
|
| 486 |
+
|
| 487 |
+
|
| 488 |
+
# --- Orchestration ------------------------------------------------------------
|
| 489 |
+
|
| 490 |
+
|
| 491 |
+
@dataclass
|
| 492 |
+
class Report:
|
| 493 |
+
failed: list[str] = field(default_factory=list)
|
| 494 |
+
skipped: list[str] = field(default_factory=list)
|
| 495 |
+
|
| 496 |
+
|
| 497 |
+
def run_step(report: Report, label: str, step: Callable[[], object]) -> object:
|
| 498 |
+
try:
|
| 499 |
+
return step()
|
| 500 |
+
except Exception as exc: # one failed format must not stop the others
|
| 501 |
+
print(f' ✗ {label} failed: {exc}')
|
| 502 |
+
report.failed.append(label)
|
| 503 |
+
return None
|
| 504 |
+
|
| 505 |
+
|
| 506 |
+
def build(recipe: Recipe, formats: set[str], out: Path, cache: Path, calib_dir: Path,
|
| 507 |
+
calib_samples: Optional[int], verify_coreml: bool, report: Report) -> None:
|
| 508 |
+
print('\n' + '=' * 70 + f'\n{recipe.name} ({recipe.folder}, {recipe.imgsz}×{recipe.imgsz})\n' + '=' * 70)
|
| 509 |
+
if recipe.note:
|
| 510 |
+
print(f' {recipe.note}')
|
| 511 |
+
warn_toolchain(recipe)
|
| 512 |
+
onnx_path: Optional[Path] = None
|
| 513 |
+
if formats & {'onnx', 'openvino'} and (recipe.onnx or recipe.onnx_verbatim):
|
| 514 |
+
result = run_step(report, f'{recipe.name} onnx', lambda: build_onnx(recipe, out, cache))
|
| 515 |
+
onnx_path = result if isinstance(result, Path) else None
|
| 516 |
+
if 'coreml' in formats and recipe.coreml is not None:
|
| 517 |
+
if sys.platform != 'darwin':
|
| 518 |
+
print(' ⊘ CoreML needs macOS')
|
| 519 |
+
report.skipped.append(f'{recipe.name} coreml')
|
| 520 |
else:
|
| 521 |
+
run_step(report, f'{recipe.name} coreml', lambda: build_coreml(recipe, out, cache, verify_coreml))
|
| 522 |
+
if 'openvino' in formats and recipe.openvino and onnx_path is not None:
|
| 523 |
+
samples = calib_samples or recipe.calib_samples or 300
|
| 524 |
+
skipped = run_step(report, f'{recipe.name} openvino',
|
| 525 |
+
lambda: build_openvino(recipe, onnx_path, out, calib_dir, samples))
|
| 526 |
+
report.skipped.extend(skipped if isinstance(skipped, list) else [])
|
| 527 |
+
if 'tflite' in formats and recipe.tflite:
|
| 528 |
+
run_step(report, f'{recipe.name} tflite', lambda: build_tflite(recipe, out, cache))
|
| 529 |
+
|
| 530 |
+
|
| 531 |
+
def adhoc_recipe(args: argparse.Namespace) -> Recipe:
|
| 532 |
+
"""A recipe from the command line, for a size/resolution the mirror does not host."""
|
| 533 |
+
precisions = [p for p in (args.openvino or 'fp16,int8').split(',') if p]
|
| 534 |
+
bad = [p for p in precisions if p not in ('fp32', 'fp16', 'int8')]
|
| 535 |
+
if bad:
|
| 536 |
+
sys.exit(f'--openvino accepts fp32, fp16, int8; got {bad}')
|
| 537 |
+
name = args.name or f'camstack-{args.model}-{args.imgsz}'
|
| 538 |
+
return Recipe(
|
| 539 |
+
name=name, folder=args.folder, imgsz=args.imgsz,
|
| 540 |
+
toolchain=Toolchain('any', 'any', 'any', 'any'), checkpoint=args.model,
|
| 541 |
+
onnx=OnnxExport(dynamic=args.dynamic, opset=args.opset),
|
| 542 |
+
coreml=CoremlExport(dynamic=args.dynamic, batch=16 if args.dynamic else 1, half=args.half),
|
| 543 |
+
openvino=tuple(OpenvinoOutput(f'-{p}', p) for p in precisions),
|
| 544 |
+
calib_samples=args.calib_samples or 300,
|
| 545 |
+
)
|
| 546 |
+
|
| 547 |
+
|
| 548 |
+
def hosted_weight_paths() -> list[str]:
|
| 549 |
+
url = f'https://huggingface.co/api/models/{HF_REPO}/tree/main?recursive=true'
|
| 550 |
+
with urllib.request.urlopen(url, timeout=60) as resp:
|
| 551 |
+
tree = json.load(resp)
|
| 552 |
+
prefixes = tuple(f'{f}/' for f in FAMILIES.values())
|
| 553 |
+
return sorted(
|
| 554 |
+
e['path'] for e in tree
|
| 555 |
+
if e['type'] == 'file' and e['path'].startswith(prefixes) and not e['path'].endswith('README.md')
|
| 556 |
+
)
|
| 557 |
+
|
| 558 |
+
|
| 559 |
+
def check_hosted() -> int:
|
| 560 |
+
"""Every hosted AGPL weight file is an output of a recipe, and vice versa."""
|
| 561 |
+
hosted = hosted_weight_paths()
|
| 562 |
+
outputs = [o for r in RECIPES for o in r.outputs()]
|
| 563 |
+
|
| 564 |
+
def covered(path: str) -> bool:
|
| 565 |
+
return any(path == o or (o.endswith('/') and path.startswith(o)) for o in outputs)
|
| 566 |
+
|
| 567 |
+
def present(output: str) -> bool:
|
| 568 |
+
return any(h == output or (output.endswith('/') and h.startswith(output)) for h in hosted)
|
| 569 |
+
|
| 570 |
+
orphans = [h for h in hosted if not covered(h)]
|
| 571 |
+
unhosted = [o for o in outputs if not present(o)]
|
| 572 |
+
print(f'{len(hosted)} hosted files under {", ".join(FAMILIES.values())}; '
|
| 573 |
+
f'{len(RECIPES)} recipes, {len(outputs)} outputs')
|
| 574 |
+
for h in orphans:
|
| 575 |
+
print(f' NOT REPRODUCED by any recipe: {h}')
|
| 576 |
+
for o in unhosted:
|
| 577 |
+
print(f' recipe output not hosted: {o}')
|
| 578 |
+
if orphans or unhosted:
|
| 579 |
+
return 1
|
| 580 |
+
print('OK: every hosted file is a recipe output, and every recipe output is hosted.')
|
| 581 |
+
return 0
|
| 582 |
+
|
| 583 |
+
|
| 584 |
+
def list_recipes() -> None:
|
| 585 |
+
for r in RECIPES:
|
| 586 |
+
ov = ','.join(f'{o.precision}{"→" + o.suffix if o.suffix else ""}' for o in r.openvino)
|
| 587 |
+
src = r.checkpoint if isinstance(r.checkpoint, str) else r.checkpoint.url
|
| 588 |
+
onnx = 'verbatim' if r.onnx_verbatim else (
|
| 589 |
+
f'{"dynamic" if r.onnx and r.onnx.dynamic else "static"} opset{r.onnx.opset if r.onnx else ""}')
|
| 590 |
+
cm = (f'{"flex≤" + str(r.coreml.batch) if r.coreml.dynamic else "static"}'
|
| 591 |
+
f'{" fp16" if r.coreml.half else ""}') if r.coreml else '-'
|
| 592 |
+
int8 = f' int8 N={r.calib_samples}' if any(o.precision == 'int8' for o in r.openvino) else ''
|
| 593 |
+
print(f'{r.name:30s} {r.folder:28s} {r.imgsz:4d} onnx[{onnx}] coreml[{cm}] '
|
| 594 |
+
f'openvino[{ov}]{int8}{" tflite" if r.tflite else ""} '
|
| 595 |
+
f'ultralytics {r.toolchain.ultralytics} from {src}')
|
| 596 |
+
|
| 597 |
+
|
| 598 |
+
def parse_args(argv: list[str]) -> argparse.Namespace:
|
| 599 |
+
p = argparse.ArgumentParser(description=__doc__.split('\n\n')[0],
|
| 600 |
+
formatter_class=argparse.RawDescriptionHelpFormatter)
|
| 601 |
+
sel = p.add_argument_group('what to build')
|
| 602 |
+
sel.add_argument('--all', action='store_true', help='every hosted recipe')
|
| 603 |
+
sel.add_argument('--family', action='append', choices=sorted(FAMILIES), help='one mirror folder')
|
| 604 |
+
sel.add_argument('--variant', action='append', help='a hosted file stem, e.g. camstack-yolov9m-320')
|
| 605 |
+
sel.add_argument('--list', action='store_true', help='print the recipes and exit')
|
| 606 |
+
sel.add_argument('--check-hosted', action='store_true',
|
| 607 |
+
help='compare the recipes with the live mirror and exit (non-zero on a gap)')
|
| 608 |
+
sel.add_argument('--dry-run', action='store_true', help='print the files that would be built')
|
| 609 |
+
p.add_argument('--formats', default=','.join(ALL_FORMATS), help='subset of onnx,coreml,openvino,tflite')
|
| 610 |
+
p.add_argument('--out', type=Path, default=Path('exported-models'))
|
| 611 |
+
p.add_argument('--cache', type=Path, default=Path('exported-models/.downloads'),
|
| 612 |
+
help='where third-party checkpoints are downloaded')
|
| 613 |
+
p.add_argument('--calib-dir', type=Path, default=DEFAULT_CALIB_DIR)
|
| 614 |
+
p.add_argument('--verify-coreml', action='store_true',
|
| 615 |
+
help='predict N=1 and N=4 on each flexible-batch CoreML package (in a child process)')
|
| 616 |
+
p.add_argument('--calib-samples', type=int, help="override the recipe's INT8 calibration image count")
|
| 617 |
+
adhoc = p.add_argument_group('ad-hoc build (not a hosted file)')
|
| 618 |
+
adhoc.add_argument('--model', help='Ultralytics id, e.g. yolo26n, yolov9m, yolo26s-seg')
|
| 619 |
+
adhoc.add_argument('--imgsz', type=int, default=640)
|
| 620 |
+
adhoc.add_argument('--name', help='output stem (default camstack-<model>-<imgsz>)')
|
| 621 |
+
adhoc.add_argument('--folder', default='custom')
|
| 622 |
+
adhoc.add_argument('--dynamic', action='store_true', help='dynamic axes (ONNX) / flexible batch (CoreML)')
|
| 623 |
+
adhoc.add_argument('--opset', type=int, default=17)
|
| 624 |
+
adhoc.add_argument('--half', action='store_true', help='FP16 CoreML')
|
| 625 |
+
adhoc.add_argument('--openvino', help='IR precisions, comma-separated: fp32,fp16,int8')
|
| 626 |
+
return p.parse_args(argv)
|
| 627 |
+
|
| 628 |
+
|
| 629 |
+
def select(args: argparse.Namespace) -> list[Recipe]:
|
| 630 |
+
if args.model:
|
| 631 |
+
return [adhoc_recipe(args)]
|
| 632 |
+
by_name = {r.name: r for r in RECIPES}
|
| 633 |
+
unknown = [v for v in (args.variant or []) if v not in by_name]
|
| 634 |
+
if unknown:
|
| 635 |
+
sys.exit(f'unknown --variant {unknown}; see --list')
|
| 636 |
+
folders = {FAMILIES[f] for f in (args.family or [])}
|
| 637 |
+
return [r for r in RECIPES
|
| 638 |
+
if args.all or r.folder in folders or r.name in (args.variant or [])]
|
| 639 |
+
|
| 640 |
+
|
| 641 |
+
def main(argv: list[str]) -> int:
|
| 642 |
+
args = parse_args(argv)
|
| 643 |
+
if args.list:
|
| 644 |
+
list_recipes()
|
| 645 |
+
return 0
|
| 646 |
+
if args.check_hosted:
|
| 647 |
+
return check_hosted()
|
| 648 |
+
recipes = select(args)
|
| 649 |
+
if not recipes:
|
| 650 |
+
print('Nothing selected: pass --all, --family, --variant or --model (see --help).')
|
| 651 |
+
return 2
|
| 652 |
+
formats = {f for f in args.formats.split(',') if f}
|
| 653 |
+
if formats - set(ALL_FORMATS):
|
| 654 |
+
sys.exit(f'--formats accepts {",".join(ALL_FORMATS)}')
|
| 655 |
+
if args.dry_run:
|
| 656 |
+
for r in recipes:
|
| 657 |
+
for o in r.outputs():
|
| 658 |
+
print(args.out / o)
|
| 659 |
+
return 0
|
| 660 |
+
report = Report()
|
| 661 |
+
for r in recipes:
|
| 662 |
+
build(r, formats, args.out, args.cache, args.calib_dir, args.calib_samples,
|
| 663 |
+
args.verify_coreml, report)
|
| 664 |
+
print('\nDone.' + (f' Failed: {report.failed}.' if report.failed else '')
|
| 665 |
+
+ (f' Skipped: {report.skipped}.' if report.skipped else ''))
|
| 666 |
+
return 1 if report.failed else 0
|
| 667 |
+
|
| 668 |
+
|
| 669 |
+
if __name__ == '__main__':
|
| 670 |
+
sys.exit(main(sys.argv[1:]))
|