Object Detection
ONNX
GGUF
agriculture
crop
plant-disease
pest
edge
on-device
onnxruntime-web
yolo11
mobilenetv3
field-images
decision-model
feature-extraction
Instructions to use CopyleftCultivars/agri-vision-detector with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use CopyleftCultivars/agri-vision-detector with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf CopyleftCultivars/agri-vision-detector:F16 # Run inference directly in the terminal: llama cli -hf CopyleftCultivars/agri-vision-detector:F16
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf CopyleftCultivars/agri-vision-detector:F16 # Run inference directly in the terminal: llama cli -hf CopyleftCultivars/agri-vision-detector:F16
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf CopyleftCultivars/agri-vision-detector:F16 # Run inference directly in the terminal: ./build/bin/llama cli -hf CopyleftCultivars/agri-vision-detector:F16
Use Docker
docker model run hf.co/CopyleftCultivars/agri-vision-detector:F16
- Llama new
- LM Studio
- Jan
- Ollama
How to use CopyleftCultivars/agri-vision-detector with Ollama:
ollama run hf.co/CopyleftCultivars/agri-vision-detector:F16
- Unsloth Desktop
- Docker Model Runner
How to use CopyleftCultivars/agri-vision-detector with Docker Model Runner:
docker model run hf.co/CopyleftCultivars/agri-vision-detector:F16
- Lemonade
How to use CopyleftCultivars/agri-vision-detector with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull CopyleftCultivars/agri-vision-detector:F16
Run and chat with the model
lemonade run user.agri-vision-detector-F16
List all available models
lemonade list
- Atomic Chat
File size: 15,567 Bytes
f79b7a1 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 | # HOWTO: reproduce, run, and extend
Everything here was run on one Dell T7610 with Tesla M40s (sm_52), torch 1.13.1, on 2026-10-08. Paths below are the ones used; change `ROOT` as needed.
Scripts live in this repo; bulky data lives under `/mnt/store1/agri` (not in git).
## 0. Ground rules learned the hard way
- **No Hugging Face transformers `generate()` on old GPUs** (it has crashed this box). Nothing here uses transformers: the detector is a CNN (Ultralytics) and d1 runs in llama.cpp.
- On a shared GPU box, reserve first (`gpusched reserve --gpu 0 ...`), pin to one GPU/NUMA node (`gpu0run`), watch temperature with a guard scoped to **your own PID**, and release when done.
- Never `pkill -f`/`pgrep -f` a pattern that your own shell command contains (it kills your shell). Match on `readlink /proc/PID/exe`, a PID you saved, or a listening port.
- Large downloads: single-stream `curl -L -C -` with retries (HF xet paths failed on this box). Check completeness by size/MD5, not exit code. Re-download if a "CSV" is a few hundred bytes (it was a redirect page: use `curl -L`).
- Disk filled to 98%: download, downscale (max side 640), record in a manifest, delete the raw zip, repeat.
## 1. Environments
```bash
python3 -m venv ~/venvs/agri-vision && ~/venvs/agri-vision/bin/pip install boto3 pycocotools pillow numpy requests tqdm # data + analysis, no torch
# detector training: reuse an existing torch 1.13.1+cu117 venv through a .pth file (so pip never touches torch), then add torchvision + ultralytics
python -m venv ~/venvs/agri-yolo && echo "<path-to>/research-pt113/lib/python3.10/site-packages" > ~/venvs/agri-yolo/lib/python3.10/site-packages/zz_pt113.pth
~/venvs/agri-yolo/bin/pip install "torchvision==0.14.1" --index-url https://download.pytorch.org/whl/cu117
~/venvs/agri-yolo/bin/pip install "ultralytics==8.3.0" opencv-python-headless onnx onnxruntime
```
Confirm after installing: `pip list | grep -i '^torch '` still shows 1.13.1 in the shared venv.
## 2. Artemis / Image Safari data (anonymous S3)
```bash
python s3_list.py ImageSafari/ / 50 # list crops; bucket alliance-artemis-imagesafari, us-west-2, unsigned
python enum_annotations.py # -> annotation_index.json (all COCO JSONs + image dirs per crop)
python fetch_coco.py # downloads box-bearing COCO files (<130 MB each) to /mnt/store1/agri/coco, prints classes
```
Facts to know: the real prefix layout differs from the README (`<crop>/annotations/<variety>/standard/<task>/<set>/`); only bean, cowpea and sorghum have true box-detection sets;
several annotation sets are duplicates (Safari bush bean == Artemis bush bean, two identical cowpea JSONs, two overlapping sorghum JSONs). `cowpea_plant` boxes include tiny noisy boxes.
Always draw boxes on a few images (`overlay.py`) before training.
## 3. Detector
```bash
python build_dataset.py /mnt/store1/agri/ds_v1 # YOLO-format dataset (3 classes), EXIF-free raw pixels, longest side <=1280
python resplit.py # per-crop, date/batch-grouped 70/15/15 split (the first greedy split was class-skewed; check class counts per split!)
gpusched reserve --gpu 0 --duration 5h --session <you> --harness claude-code --purpose "YOLO11n detector"
./run_train.sh yolo11n_v1 60 1.0 640 16 # smoke test first: ./run_train.sh smoke 1 0.05 640 16 (fraction takes the first N files alphabetically)
python eval_detector.py yolo11n_v1 # CPU eval on val+test, torch vs ONNX -> detector_eval_yolo11n_v1.json
```
Settings: YOLO11n pretrained, imgsz 640, batch 16, `amp=False` (M40 has no fast fp16), patience 15. ~2.2 min/epoch; early-stopped at epoch 55 (best 40). GPU peaked ~83 C.
`thermal_guard.sh PID GPU LIMIT LOGFILE` kills only that PID after three consecutive reads at/above the limit.
### Run the exported model
```bash
python infer_onnx.py best.onnx photo.jpg --conf 0.3 --iou 0.5 --save out.jpg # onnxruntime + numpy + PIL only
```
Output tensor `(1, 7, 8400)`: cx, cy, w, h (letterboxed 640 px) + scores for `common_bean, cowpea, sorghum`. For Android use ONNX Runtime Mobile (not tested here).
TFLite: **does not work yet.** `onnx2tf` 1.22.3 + TF 2.15 failed on YOLO11n (detect head, then C2PSA attention) and YOLOv8n (a Conv op). Don't spend long on it without a newer toolchain.
## 4. Disease / pest data
```bash
python disease_manifest.py plantdoc|plantwild|plantseg|ip102|cassava # per-source parser: downscale to max 640, append rows to disease_manifest.jsonl
python make_crop_set.py # ~100 Safari field images per crop for crop-ID (excludes probe images by exact key AND by image-hash stem)
python canon.py # prints how raw condition strings map to the 12 coarse concepts
```
Sources and exact URLs: PlantDoc `codeload.github.com/pratikkayal/PlantDoc-Dataset/zip/refs/heads/master`; PlantWild `huggingface.co/datasets/uqtwei2/PlantWild/resolve/main/plantwild.zip`;
PlantSeg `zenodo.org/api/records/16665770/files/Plantseg.zip/content` (MD5 23385d877bc84bab51b54630055c75b8); IP102 COCO `zenodo.org/api/records/15546419/files/ip102.zip/content`;
Cassava via HF `Engineer101/cassava-leaf-disease-classification` (individual files; use the repo's file list to find `train_images/part_xxx/`).
**Audit labels by eye** (we did: contact sheets with the mapped label printed on each tile). Findings: PlantWild is noisy; PlantDoc healthy = white-background stock images (style confound).
Leakage checks that mattered: the same photo appeared under several annotation sets (8 training images were probe images; 14 within-set duplicates) — compare image-hash stems, not only keys.
## 5. d1-omni-600M locally (llama.cpp) and the trunk-feature patch
```bash
git clone https://github.com/ggml-org/llama.cpp /mnt/store1/agri/llama.cpp-d1 # tested at de7fa0a (2026-10-08); older checkouts lack /v1/systemone
cmake -B build-cuda -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=52 -DLLAMA_CURL=OFF && cmake --build build-cuda -j6 --target llama-server
# models: curl -L -C - the d1-omni-600M-{Q8_0,F16}.gguf and mmproj-d1-omni-600M-{Q8_0,F16}.gguf files from LiquidAI/d1-omni-600M-GGUF
D1_EXPORT_TRUNK=1 D1_DUMP_PATH=/tmp/dump.bin gpu0run ./build-cuda/bin/llama-server -m d1-omni-600M-F16.gguf --mmproj mmproj-d1-omni-600M-F16.gguf \
--port 8097 -b 4096 -ub 4096 -ngl 99 -np 1 --image-max-tokens 256
```
**The patch** (two small edits, kept outside upstream; `patches/d1-trunk-export.patch`, 17 added lines, verified to apply cleanly to llama.cpp commit de7fa0a): (1) `src/models/lfm2.cpp` — when `D1_EXPORT_TRUNK` is set, `n_embd_out` grows from 3 to 3+1024 and the post-`output_norm`
trunk state at the output rows is concatenated after the 3 score columns; (2) `tools/server/server-context.cpp` — when `D1_DUMP_PATH` is set, append a record `[int32 n_markers, int32 n_embd_out, float32 x n_markers*n_embd_out]` per request.
Requires `-np 1` (records are matched to requests by order). `extract_features.py` asserts the record count/order and that `softmax(dumped logits)` reproduces the server's returned probabilities.
Questions are sent to `POST /v1/systemone` with `files:[data-URL image]` and `questions:{id:{type:choice|score|noul, instructions, criteria}}`. `noul` returns P(true). Throughput on one M40: ~0.45 s/image at 448 px.
CPU-only is ~190 s for a 1080p image: use a GPU.
### Probe and feature extraction
```bash
python make_probe_set.py && python sheets.py # 204-image probe + numbered contact sheets (look at them, then fill probe_flags.json)
python probe_run.py http://127.0.0.1:8099 probe_results_f16.jsonl 896 && python probe_analyze.py probe_results_f16.jsonl
python build_extract_lists.py && for n in probe artemis_det safari_crop plantdoc cassava plantwild plantseg ip102; do python style_feats.py $n; done
./run_extraction.sh # unattended: reserves GPU, starts the patched server under a thermal guard, extracts each list, tears down
```
## 6. Heads
```bash
python train_head.py --tasks crop,healthy,cond,pest # plain PyTorch on CPU (~2 min); every number next to d1 zero-shot, a style-only control and majority class, with Wilson 95% CIs
python export_heads.py # refit on train+val, save artifacts/heads_v1/*.npz (+json), verify numpy forward == torch
```
**Dedup is mandatory.** PlantDoc, PlantWild and PlantSeg contain the same photos (65 of 2,578 PlantDoc images match at cosine >= 0.98 in d1 trunk-feature space; verified by eye). `train_head.py` flags any evaluation image with a >= 0.95 near-duplicate in the training pool (`flagdups`) and drops it from evaluation only. `train_head_v1_before_dedup.py` is the pre-dedup version, kept so the effect is auditable (up to ~4 points).
The saved heads are plain NumPy: `logits = ((x - mu) / sd) @ W.T + b` on the 3,072-d feature vector.
Rules we hold ourselves to: report n and a Wilson interval for every rate; compare with zero-shot and with a no-semantics control; evaluate on a different source/campaign than training
(PlantDoc is the cross-source test; Artemis detection images are the out-of-domain crop test); hand-check flagged samples before quoting a count.
## 7. Backups
Features, manifests and weights are not "recreatable": they depend on the exact d1 build, patch and hardware. Keep a private copy with a SHA-256 manifest and re-hash the copy.
`artifacts/yolo11n_v1/SHA256SUMS` is an example (copies verified byte-identical).
## 8. Android (status: builds only)
```bash
cmake -B build-android -DCMAKE_TOOLCHAIN_FILE=$NDK/build/cmake/android.toolchain.cmake -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-28 -DCMAKE_BUILD_TYPE=Release \
-DGGML_OPENMP=OFF -DGGML_NATIVE=OFF -DLLAMA_CURL=OFF -DLLAMA_OPENSSL=OFF -DBUILD_SHARED_LIBS=OFF && cmake --build build-android -j3 --target llama-server
```
Produced a 185 MB (unstripped) aarch64 ELF with NDK r27c. **Never executed on a device**; latency and memory on a phone are unmeasured. Expect CPU-only (no GPU offload) and test the Q8_0 model (407 MB + 263 MB mmproj).
## 9. End-to-end demo (CPU only, no GPU needed)
```bash
# CPU-only build of the patched server (never loads CUDA): cmake -B build-cpu -DGGML_CUDA=OFF -DLLAMA_CURL=OFF && cmake --build build-cpu --target llama-server
D1_EXPORT_TRUNK=1 D1_DUMP_PATH=/tmp/dump.bin ./build-cpu/bin/llama-server -m d1-omni-600M-F16.gguf --mmproj mmproj-d1-omni-600M-F16.gguf --port 8097 -np 1 -t 6 --image-max-tokens 256 &
python diagnose.py photo.jpg --server http://127.0.0.1:8097 --dump /tmp/dump.bin --save out.jpg # detector boxes + crop / healthy / condition head outputs as JSON, with warnings
```
~6-9 s per image on 6 CPU threads (d1 dominates; the ONNX detector is fast). Features from this CPU path matched the stored GPU-extracted features (cosine 1.00000, relative L2 difference ~0.001 on 3 images), so CPU and GPU extraction are interchangeable here.
`diagnose.py` is a research demo: the detector only knows bean/cowpea/sorghum seedlings in Artemis-style photos and will mislabel other crops (it called a cassava leaf "sorghum").
## 10. Tiny CNN diagnoser (no d1, no llama.cpp) — added 2026-10-10
Everything below is plain PyTorch/torchvision/ONNX Runtime (no transformers), run on the same M40 box via the same `gpusched` + `gpu{0,1}run` + scoped thermal-guard protocol as section 0 (`run_cnn.sh GPU NAME args...` does the pinning and guard).
```bash
# 1. caches (uint8 224x224 squash-resize, row order == meta order; ~8 GB for the disease manifest, ~4 GB CCMT)
python build_img_cache.py 224 # disease_manifest.jsonl minus IP102 -> imgcache/img224.npy, meta.jsonl
python ccmt_manifest.py && python build_cache_generic.py imgcache/ccmt_meta.jsonl imgcache/ccmt 224 # CCMT raw .jpg (see notebook 31: names collide across folders, fetch with uuid suffix)
python ip102full_manifest.py && python build_cache_generic.py imgcache/ip102full_meta.jsonl imgcache/ip102full 224
# 2. train (MobileNetV3-L from torchvision weights; heads: healthy/diseased + masked 12-way condition); measured on one M40: E1 (21k training images, 14 epochs) ~24 min, E5 (53k images, 14 epochs) ~65 min
./run_cnn.sh 0 e5_all_ccmt --train plantwild,plantseg,plantdoc,cassava,karaagro,makerere,pretoria,ccmt --epochs 14 [--seed N] [--arch effb0] [--aug strong]
# leave-one-source-out: simply leave a source out of --train; every source not in --train is evaluated as an external test (test split; PlantDoc/CCMT all rows), near-dups excluded
# 3. other tasks
DS=ip102full ./run_cnn_pest.sh 0 pest_ip102full 30 # 97-class pest species, pHash-deduped test
./run_cnn_crop.sh 0 crop_sites 15 # site-disjoint 5-fold crop ID (Safari)
./run_cnn_ident.sh 1 species_gbif 15 gbif_amazon taxon group # observer-grouped species ID (GBIF)
# 4. analysis
python calibrate_cnn.py RUN train_srcs eval_srcs # temperature + abstention curves (T fitted on source val only)
python eval_cnn_ccmt.py RUN all # external CCMT evaluation, per class / crop
python eval_tta_ens.py run1,run2,run3 plantdoc,karaagro,ccmt # TTA / seed-ensemble check
python per_condition_eval.py RUN SRC # per-condition error analysis
# 5. export + INT8 (opset 13 needed for per-channel QDQ; PERCENTILE calibration is required - MinMax/Entropy collapse to chance)
OPSET=13 python export_cnn_onnx.py RUN && CALIB=Percentile python quantize_cnn_onnx.py RUN karaagro && python export_cnn_onnx.py RUN # last call: opset-12 fp32
# 6. run it (onnxruntime + PIL + numpy only)
python infer_tiny.py artifacts/yolo11n_ft1280/best.onnx artifacts/cnn_tiny_v1/cnn_tiny_int8_percentile.onnx photo.jpg --unsure 0.75 --save out.jpg
```
Notes: (a) the detector input size is read from the ONNX (1280 for `yolo11n_ft1280`, which must be run at 1280; 960 for v2/v1res960); (b) `infer_tiny.py` runs the classifier on each detected plant crop **and** on the whole image, because the classifier was trained on leaf/close-up and whole-plant photos, not on tiny Safari crops: crops below 24 px are reported as `too_small`; (c) confidences are raw softmax, not calibrated (notebook 33); (d) the trained model has never seen a hemp image and was not evaluated on Amazonian crops for disease.
Detector fine-tune used for the 1280 model: `./run_train.sh yolo11n_ft1280 25 1.0 1280 8 <v1res960/best.pt> 0.002` (GPU0, 3.5 h). Evaluate with `DEV=0 python eval_detector.py yolo11n_ft1280 1280 ft1280` and `DEV=0 python size_recall.py <weights.pt> 1280 ft1280` (model must be a path).
## 11. Crop ID with web photos (added 2026-10-25)
```bash
python3 -I gbif_fetch.py /mnt/store1/agri/dl/gbif_crops 150 "Zea mays" "Sorghum bicolor" ... # GBIF/iNaturalist photos
# build imgcache/gbif_crops_meta.jsonl (add a "crop" field per taxon), then:
python build_cache_generic.py imgcache/gbif_crops_meta.jsonl imgcache/gbif_crops 224
./run_web_chain.sh # matched web vs control (MAXW=0), seeds 0-2, GPU0, ~25 min each; hold a gpusched reservation first
SEED=0 gpu0run python -I train_crop_final.py crop18_web_final 15 # final all-data model (~7 min)
python -I export_crop18_onnx.py artifacts/crop18_web_v1 # ONNX opset 12 + parity check (CPU)
```
Web rows are train-only (fold=-1); the test set is computed from Safari rows so it is identical with and without web photos.
|