Object Detection
ONNX
GGUF
agriculture
crop
plant-disease
pest
edge
on-device
onnxruntime-web
yolo11
mobilenetv3
field-images
decision-model
feature-extraction
Instructions to use CopyleftCultivars/agri-vision-detector with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use CopyleftCultivars/agri-vision-detector with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf CopyleftCultivars/agri-vision-detector:F16 # Run inference directly in the terminal: llama cli -hf CopyleftCultivars/agri-vision-detector:F16
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf CopyleftCultivars/agri-vision-detector:F16 # Run inference directly in the terminal: llama cli -hf CopyleftCultivars/agri-vision-detector:F16
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf CopyleftCultivars/agri-vision-detector:F16 # Run inference directly in the terminal: ./build/bin/llama cli -hf CopyleftCultivars/agri-vision-detector:F16
Use Docker
docker model run hf.co/CopyleftCultivars/agri-vision-detector:F16
- Llama new
- LM Studio
- Jan
- Ollama
How to use CopyleftCultivars/agri-vision-detector with Ollama:
ollama run hf.co/CopyleftCultivars/agri-vision-detector:F16
- Unsloth Desktop
- Docker Model Runner
How to use CopyleftCultivars/agri-vision-detector with Docker Model Runner:
docker model run hf.co/CopyleftCultivars/agri-vision-detector:F16
- Lemonade
How to use CopyleftCultivars/agri-vision-detector with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull CopyleftCultivars/agri-vision-detector:F16
Run and chat with the model
lemonade run user.agri-vision-detector-F16
List all available models
lemonade list
- Atomic Chat
|
Download docs/MODEL_CARD_DETAILED.md from CopyleftCultivars/agri-vision-detector: direct link, hf CLI and curl.
- Browser
- Download file 18.1 kB
-
https://huggingface.co/CopyleftCultivars/agri-vision-detector/resolve/main/docs/MODEL_CARD_DETAILED.md
- Command line
-
hf download hf://CopyleftCultivars/agri-vision-detector/docs/MODEL_CARD_DETAILED.md
-
curl -L -o MODEL_CARD_DETAILED.md https://huggingface.co/CopyleftCultivars/agri-vision-detector/resolve/main/docs/MODEL_CARD_DETAILED.md
18.1 kB
| # Detailed measurements (sections 1-8) | |
| Moved from the first card draft. Model licensing is in the repo README and LICENSE files. Numbering kept so older references still work. | |
| ## 1. Quick start (detector, no deep-learning framework needed) | |
| ```bash | |
| pip install onnxruntime pillow numpy | |
| python infer_onnx.py best.onnx field_photo.jpg --conf 0.3 --iou 0.5 --save out.jpg | |
| ``` | |
| `infer_onnx.py` is in this repo: letterbox to 640x640, run, decode `(1, 7, 8400)` (cx, cy, w, h in letterboxed pixels + 3 class scores), per-class NMS. | |
| Tested against Ultralytics' own predictor on 12 held-out test images: top-detection confidences matched (e.g. 0.92 vs 0.92) and detection counts at conf 0.3 | |
| were within +/-1 on 10 of 12 images (the rest differed by up to 5 because of PIL-vs-OpenCV resizing near the threshold). With Ultralytics installed: | |
| `YOLO("best.onnx").predict("field_photo.jpg", imgsz=640, conf=0.3)`. | |
| ## 2. Detector results (held-out) | |
| | model / split | images | mAP50 | mAP50-95 | precision | recall | | |
| |---|---|---|---|---|---| | |
| | PyTorch, val | 1,227 | 0.581 | 0.294 | 0.648 | 0.558 | | |
| | **PyTorch, test** | 1,246 | **0.644** | **0.349** | 0.681 | 0.580 | | |
| | ONNX, test | 1,246 | 0.641 | 0.346 | 0.675 | 0.575 | | |
| Per-class test AP50-95: common_bean 0.328, cowpea 0.366, sorghum 0.352. Evaluated at conf 0.001 / IoU 0.6. Val was used for early stopping; **test is the unbiased number.** | |
| Splits are by capture date (or source batch when no date is recoverable), per crop, so near-duplicate adjacent frames do not straddle train and test. | |
| All three crops appear in every split. The 987 climbing-bean images have no recoverable date, so they are train-only; bean val/test are bush beans. | |
| What the numbers hide (from looking at predictions overlaid on test images): | |
| - **Tiny / distant plants are missed.** One drip-row photo with 22 labelled ~15 px plants got 1 detection at 640 px input. | |
| - **Labels are subjective and incomplete.** Box boundaries around overlapping plants vary, and some visible plants/weeds are unlabelled, so mAP50-95 understates quality on medium-sized plants and precision is understated by missing labels. | |
| ## 3. What d1 can and cannot do zero-shot (honest baseline) | |
| `d1-omni-600M` answers typed multiple-choice / yes-no questions about an image in one forward pass (no text generation). On 204 field photos (12 crops x 17, drawn from Image Safari; | |
| I checked each image on contact sheets and flagged 13 as ambiguous): | |
| | setting | crop top-1 (excl. 13 ambiguous) | Wilson 95% | | |
| |---|---|---| | |
| | d1 zero-shot, F16, 896 px | 40/191 = 20.9% | [15.8, 27.3] | | |
| | d1 zero-shot, Q8_0, 896 px | 39/191 = 20.4% | [15.3, 26.7] | | |
| | d1 zero-shot, F16, 448 px (the feature-extraction setting) | 40/191 = 20.9% | | | |
| Only banana (17/17) and maize (15/16) are recognised. Most other crops are called "maize" (96 of 164 wrong answers had confidence > 0.9). This is not option-order bias: | |
| reversing the option order and adding descriptions gave 8/60 with 41/60 still "maize". Q8_0 and F16 agree on 199/204 crop answers, so quantising is not the problem. | |
| The zero-shot disease yes/no score was not usable (no ground truth exists in this dataset; of the 12 images it rated most diseased I judged 2 plausible, 5 ambiguous, 5 healthy-looking). | |
| **This is why the heads in section 4 exist.** | |
| ## 4. d1 heads (Tier A: frozen trunk + small trained head) | |
| Method: a local patch to llama.cpp's `llama-server` (env `D1_EXPORT_TRUNK=1`) appends d1's 1,024-d trunk state to each option's output, so we cache one | |
| 3,072-d vector per image (mean trunk state for each of three fixed questions: crop, condition concept, plant part) plus the options' 42 raw logits, and train a | |
| logistic-regression/MLP head in plain PyTorch on CPU. No fine-tuning of d1's weights; no transformers library involved. | |
| Every number below is reported next to (a) d1 zero-shot on the same images and (b) a **style-only control** (colour grid, white-background fraction, edge density, aspect | |
| ratio; no semantics), because healthy images in PlantDoc are mostly white-background stock photos and would otherwise inflate healthy-vs-diseased accuracy. | |
| All numbers: accuracy with Wilson 95% CI, **after removing evaluation images that have a near-duplicate (cosine >= 0.95 on d1 trunk features) in the training pool**. | |
| Heads use the 3,072-d trunk features ("feat"); the "logit" and "feat+logit" variants are in `head_all_dedup.log` and were not better. Hyper-parameters were chosen on validation splits only. | |
| | task (train -> test) | n | **d1 trunk head** | d1 zero-shot | style-only control | majority class | | |
| |---|---|---|---|---|---| | |
| | **Crop ID, 17 crops**, Safari train -> held-out Safari probe (same campaign) | 187 | **91.4%** [86.6, 94.7] | 18.7% [13.8, 24.9] | 27.8% [21.9, 34.6] | 0% | | |
| | Crop ID, Safari train -> **Artemis detection images** (different campaign/camera; bean, cowpea, sorghum) | 299 | **31.1%** [26.1, 36.6] | 0.3% | 0% | 0% | | |
| | **Healthy vs diseased**, PlantWild train -> PlantWild test | 644 | **87.4%** [84.6, 89.8] | 80.3% [77.0, 83.2] | 70.7% [67.0, 74.0] | 59.0% | | |
| | Healthy vs diseased, PlantWild train -> **PlantDoc** (cross-source) | 2,523 | **90.8%** [89.6, 91.9] | 84.7% [83.3, 86.1] | 72.1% [70.4, 73.9] | 66.7% | | |
| | **Condition (9 coarse classes)**, PlantWild+PlantSeg train -> PlantWild test | 639 | **68.1%** [64.4, 71.6] | 43.5% | 30.8% | 41.3% | | |
| | Condition, -> PlantSeg test | 478 | **60.5%** [56.0, 64.7] | 11.5% | 21.5% | 0% | | |
| | Condition, -> **PlantDoc** (cross-source) | 2,521 | **63.4%** [61.5, 65.2] | 36.6% [34.8, 38.5] | 26.7% | 33.3% | | |
| | **Pest species (IP102, 102 classes)**, official split sample | 534 | **40.1%** [36.0, 44.3] (chance ~1%) | n/a (no species options) | 10.9% | 0.6% | | |
| Honest reading: | |
| - **The heads help a lot over d1 zero-shot**, and the style-only control (colours, white background, edges) is far below them, so they are not just learning photo style. | |
| For healthy-vs-diseased the gain over zero-shot is modest (+6 points on PlantDoc) because d1's "healthy green leaves" concept already works reasonably zero-shot. | |
| - **Crop ID does not generalise across campaigns.** The 91% is in-distribution. On the Artemis images the head is right for beans (85 of 100) but 1 of 100 for cowpea and 8 of 100 for sorghum (checked on the pre-dedup test set): it calls | |
| almost everything "common bean" (an always-bean classifier would score ~33%, the head scores 31%). Treat crop ID as unsolved outside the Safari collection. | |
| - **Duplicates were real.** The same photos appear in PlantDoc, PlantWild and PlantSeg (65 of 2,578 PlantDoc images matched at cosine >= 0.98, several verified by eye as identical). Removing near-duplicates | |
| changed accuracies by up to ~4 points (largest: PlantSeg test 64.6% -> 60.5%), and without that check the cross-source numbers would have been optimistic. The 0.95 threshold only catches near-identical images; re-cropped copies can slip through. | |
| - **Style confound partly present.** For the feat+logit variant of the healthy-vs-diseased head, PlantDoc accuracy is 96.1% on white-background images (n=362) vs 90.3% on the rest (n=2,027, balanced accuracy 0.84), so stock-photo style does help, but the head is clearly above the style-only control (72%) on the non-white subset too. | |
| - **Label quality caps these numbers.** PlantWild is web-scraped and noisy; the coarse condition classes are my keyword mapping of 74 raw disease names (e.g. "leaf mold" -> mildew class; "leaf curl" dropped as ambiguous); | |
| cassava is evaluated separately below. | |
| - Small test sets (e.g. ~5 images per pest species); intervals above are wide where n is small. | |
| **Cassava (Uganda field photos; the closest proxy we have for the African target)** — features extracted, evaluated separately (`eval_cassava.log`, `eval_multisource.log`): | |
| | test | n | result | comparison | | |
| |---|---|---|---| | |
| | 5-class cassava head (bacterial blight / brown streak / green mottle / mosaic / healthy), trained on cassava | 300 | **60.3%** [54.7, 65.7] | style-only 39.0%, majority 20% | | |
| | PlantWild-trained healthy-vs-diseased head applied cross-domain to all cassava (400 healthy / 1,600 diseased) | 2,000 | balanced accuracy **0.605** (healthy recall 71%, diseased recall 50%) | | |
| | saved `condition9` head, grouped healthy / blight / virus | 2,000 | 46.4% (balanced 0.553) | zero-shot 24.9% (balanced 0.414); majority 60% | | |
| | healthy-vs-diseased head trained on PlantWild **+ cassava train** | 300 | 83.3% (balanced **0.771**) | PlantWild-only head on the same test: 63.0% (balanced 0.625); PlantDoc 89.7% vs 90.8%, PlantWild test unchanged | | |
| Reading: **heads trained on web/lab-style leaf photos do not transfer to field canopy photos** (cross-domain cassava is no better than d1 zero-shot); adding in-domain data fixes most of it without hurting the other test sets. | |
| The cassava test is a random split of one Kaggle collection (no capture dates), so in-domain numbers are optimistic, and the 300-image test is 80% diseased, so use balanced accuracy. | |
| A single end-to-end example (`diagnose.py`) on a cassava mosaic image: the condition head said mosaic/yellowing virus (74.6%, correct), but the detector called the leaf "sorghum" (0.68) and the crop head said yam/common bean: both are out-of-distribution there. | |
| **Saved heads** (`artifacts/heads_v1/*.npz`, plain NumPy: standardise with `mu`/`sd`, then `x @ W.T + b`; 50 KB - 1.2 MB each, classes in the `.json`): `crop17`, `healthy_vs_diseased`, `condition9`, `pest102`. | |
| Re-scored from the saved files on the same deduplicated test sets: crop17 probe 92.5% / Artemis 30.8%; healthy_vs_diseased PlantDoc 90.4%; condition9 PlantDoc 63.4%; pest102 42.7%. They were refit on train+val; test sets were never used for fitting. | |
| ## 4b. Tiny CNN diagnoser (end-to-end fine-tuned; added 2026-10-10) | |
| A second, much smaller classifier that needs **no d1 and no llama.cpp**: MobileNetV3-Large (ImageNet-pretrained, fine-tuned end to end in plain PyTorch on the same disease sources) with two heads, | |
| **healthy vs diseased** and a **12-way coarse condition** (same 12 concept labels as the d1 heads). 224x224 whole-image input. ONNX: **11.9 MB fp32, 3.4 MB INT8**. On a busy 2014-era Xeon core it ran at ~21 ms/image (fp32, 1 thread, ONNX Runtime); never run on a phone. | |
| Release candidate: `artifacts/cnn_tiny_v1/` (trained on PlantWild, PlantSeg, PlantDoc, Cassava, KaraAgro, Makerere, Pretoria and CCMT train splits; `SHA256SUMS` included). `infer_tiny.py` runs detector -> per-plant crop -> classifier with an "unsure" band, ONNX Runtime only. | |
| **How to read these numbers.** In-source test splits of KaraAgro, Makerere and CCMT come from random/hash splits over consecutive shots of the same fields and read ~0.97-0.99 (Pretoria's in-source split is unusable: 0.49-0.56, below its majority baseline); the same CCMT images score **0.84-0.87** accuracy (balanced 0.83-0.88) when CCMT was never trained on. Use the held-out-source rows below, not the in-source ones. | |
| | question | test (never trained on) | n | CNN | comparison | | |
| |---|---|---|---|---| | |
| | web -> web: healthy vs diseased, trained on PlantWild+PlantSeg | PlantDoc (deduped) | 2,523 | **0.943** (3 seeds: 0.939, 0.947, 0.942) | d1 trunk head, same rows: 0.915 [0.904, 0.925] | | |
| | 12-way condition, same training | PlantDoc | 2,523 | 0.780 (1 seed) | d1 head: 0.640 | | |
| | web only -> tropical field: healthy vs diseased | KaraAgro (Ghana maize) | 768 | 0.535-0.555 (3 seeds) **below the 0.694 majority baseline** | style-only 0.483 | | |
| | + other field sources in training, KaraAgro held out | KaraAgro | 768 | **0.801** [0.771, 0.828], balanced 0.805 | d1 head 0.784, balanced 0.805; style-only 0.483 | | |
| | + other field sources, Makerere held out | Makerere (Uganda maize) | 2,113 | **0.969** [0.961, 0.976] (1 seed; web-only 0.805-0.841) | d1 head on 861 rows: 0.905; style-only 0.634 | | |
| | external, never trained on | CCMT raw (Ghana; cashew/cassava/maize/tomato; 87% diseased) | 25,122 | balanced accuracy **0.83-0.88** (accuracy 0.84-0.87, equal to majority) | maize is weakest (0.63-0.76) | | |
| | pest species, 97 IP102 classes (19k images, official split, pHash-deduped test) | IP102 test | 2,969 | top-1 **0.726** [0.710, 0.742], top-5 0.914, balanced top-1 0.537 | small-sample comparison on the d1 test rows: CNN 0.530 vs d1 head 0.401 | | |
| | crop ID, 18 crops, site-disjoint folds | Safari sites | 2,824 | 0.523 [0.504, 0.541] | d1 head 0.518; style-only 0.127 | | |
| | Amazonian/tropical useful-plant species ID, 11 taxa from GBIF photos, observer-disjoint folds | GBIF | 2,238 | 0.818 [0.801, 0.833] | majority 0.134 (internal experiment) | | |
| What the CNN work showed (all logged in `LAB_NOTEBOOK.md` sections 26-46): | |
| - **Adding field data from other farms/countries is what moves field numbers**; photometric augmentation, test-time augmentation, a 3-seed ensemble and EfficientNet-B0 did not (all within seed noise or worse). | |
| - **Abstention helps, calibration does not transfer.** Taking the 80% most confident KaraAgro predictions raises balanced accuracy 0.805 -> 0.867; temperature scaling fitted on source validation data made held-out calibration worse (ECE 0.095 -> 0.117). | |
| - **INT8** (ORT static QDQ) keeps accuracy (KaraAgro-held-out model 0.814 vs 0.801 fp32) only with percentile calibration; MinMax/Entropy calibration collapsed to chance. No speed claim. | |
| - Error analysis, KaraAgro held out: maize streak virus 0.90 correct, healthy 0.82, fall-armyworm damage 0.65. | |
| - Seed-to-seed spread is 1 point on web sets but 2-4 points on Makerere/KaraAgro and up to 14 on the small Pretoria split, so single-seed field differences below that are not evidence. | |
| - A d1 head and the CNN tie on KaraAgro and on site-grouped crop ID; the CNN is better on web cross-source and on Makerere. Neither is deployment-grade for farmer-facing diagnosis. | |
| ## 5. Data and provenance | |
| | source | use | size used | | |
| |---|---|---| | |
| | Artemis / Image Safari (CIAT/Alliance Bioversity, AWS Open Data `alliance-artemis-imagesafari`) | detector training/eval; crop-ID head; d1 probe | 8,160 detection images; 1,653 crop-ID images; 204 probe images | | |
| | PlantDoc | disease head train/eval (cross-source test) | 2,578 | | |
| | PlantWild (original release) | disease head | 3,560 sampled of 18,542 | | |
| | PlantSeg | disease head | 2,792 sampled of 11,458 | | |
| | IP102 (Zenodo 15546419 COCO re-upload by a third party, not the original authors) | pest-species head (d1: 2,308 sampled) and CNN (all 18,969 images, 97 classes present) | 2,308 / 18,969 | | |
| | Cassava Leaf Disease (HF mirror `Engineer101/...`) | disease head / CNN | 2,000 (400/class) | | |
| | KaraAgro AI Maize (Ghana; Harvard Dataverse 10.7910/DVN/CXUMDS v4) | CNN and d1 head (healthy / fall armyworm / maize streak virus), held-out-source tests | 5,172 unique photos (one per base image, label derived from the VOC boxes, not folder names) | | |
| | Maize photos, Makerere AI Lab (Uganda; Harvard Dataverse 10.7910/DVN/LPGHKK) | CNN and d1 head (healthy / leaf blight / maize streak virus) | 7,421 photos, 9 capture days (class is tied to collection day: confounded) | | |
| | Pretoria "Diseases of maize in the field" (Figshare 20237613) | CNN and d1 head | 2,355 photos; split unreliable (five camera-image groups hold most images) | | |
| | CCMT, Mendeley Data bwh3zbpkpv (Ghana; cashew, cassava, maize, tomato) | CNN external test and training (raw .jpg only; the 105k augmented copies were not used) | 25,215 raw images (93 unreadable) | | |
| | GBIF occurrence photos (iNaturalist-contributed) for 11 Amazonian/tropical useful-plant taxa | internal species-ID experiment only | 2,238 photos | | |
| Not obtained: DLCPD-25 (Baidu only), AgriPest (never released), INSECT204. Hemp: no labelled public set was obtainable (notebook section 43). | |
| Images are not redistributed; manifests and scripts are in the repo, so the data can be re-fetched. (An early version of this card said CCMT was blocked; it is now fetched, see notebook sections 31-32.) | |
| ## 7. Limitations (summary) | |
| 3 of 18 crops covered by the detector; tiny-plant recall is poor (tiled inference trades precision for recall; no setting beat full-image F1); disease labels are mostly web-scraped or lab-style and noisy (e.g. a cannabis leaf appeared under "maple, healthy" in PlantWild); | |
| PlantDoc healthy/diseased differ in photographic style; cassava disease classes are visually subtle; no on-device latency measured (the patched llama.cpp server cross-compiles for Android arm64 | |
| with NDK r27c, but has never been run on a phone); TFLite export was attempted four times and failed (onnx2tf errors), so ONNX is the only mobile format. | |
| ## 8. Files | |
| `best.onnx`, `best.pt`, `results.csv`, `args.yaml`, `detector_eval_yolo11n_v1.json`, `SHA256SUMS` (copies verified byte-identical to the training outputs), `infer_onnx.py`, dataset builders and manifests, | |
| `train_detector.py`, `eval_detector.py`, `train_cnn.py`, `export_cnn_onnx.py`, `quantize_cnn_onnx.py`, `infer_tiny.py`, `artifacts/cnn_tiny_v1/`, `extract_features.py`, `train_head.py`, `export_heads.py`, `eval_cassava.py`, `eval_multisource.py`, `diagnose.py` (end-to-end demo), and the llama.cpp patch (see HOWTO.md). | |
| ## Citation / credits | |
| Artemis & Image Safari: Mutuvi S. et al., agriRxiv 2026.00491 (CIAT/Alliance of Bioversity International). d1-omni: Liquid AI. YOLO11: Ultralytics. PlantDoc, PlantWild, PlantSeg, IP102, Cassava Leaf Disease: cite per their papers. | |
| Trained on a single Tesla M40 (sm_52) with torch 1.13.1. [ORG / authors / contact: fill in] | |
| ## 4c. 18-crop identifier with web photos (added 2026-10-25) | |
| `artifacts/crop18_web_v1/` (MobileNetV3-L, 224 px, fp32 ONNX 11.9 MB, 18 Safari crops). Trained on Safari field photos plus 2,235 GBIF/iNaturalist photos (16 of 18 crops; the images themselves are not redistributed). Held-out estimate = site-disjoint 5-fold CV of the same recipe, **0.595-0.605 pooled over 3 seeds (n=2,824) vs 0.503-0.533 without web photos**; the shipped model is fit on all rows so has no test set of its own. Expect about 0.60 or lower at a new site; crops without web photos (african yam, lentil) and finger millet (38 photos) are weaker by construction. Not INT8-quantised, not run on a phone, labels not audited by an expert. Details: LAB_NOTEBOOK sections 49-50. | |