Object Detection
ONNX
GGUF
agriculture
crop
plant-disease
pest
edge
on-device
onnxruntime-web
yolo11
mobilenetv3
field-images
decision-model
feature-extraction
Instructions to use CopyleftCultivars/agri-vision-detector with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use CopyleftCultivars/agri-vision-detector with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf CopyleftCultivars/agri-vision-detector:F16 # Run inference directly in the terminal: llama cli -hf CopyleftCultivars/agri-vision-detector:F16
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf CopyleftCultivars/agri-vision-detector:F16 # Run inference directly in the terminal: llama cli -hf CopyleftCultivars/agri-vision-detector:F16
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf CopyleftCultivars/agri-vision-detector:F16 # Run inference directly in the terminal: ./build/bin/llama cli -hf CopyleftCultivars/agri-vision-detector:F16
Use Docker
docker model run hf.co/CopyleftCultivars/agri-vision-detector:F16
- Llama new
- LM Studio
- Jan
- Ollama
How to use CopyleftCultivars/agri-vision-detector with Ollama:
ollama run hf.co/CopyleftCultivars/agri-vision-detector:F16
- Unsloth Desktop
- Docker Model Runner
How to use CopyleftCultivars/agri-vision-detector with Docker Model Runner:
docker model run hf.co/CopyleftCultivars/agri-vision-detector:F16
- Lemonade
How to use CopyleftCultivars/agri-vision-detector with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull CopyleftCultivars/agri-vision-detector:F16
Run and chat with the model
lemonade run user.agri-vision-detector-F16
List all available models
lemonade list
- Atomic Chat
|
Download docs/HOWTO.md from CopyleftCultivars/agri-vision-detector: direct link, hf CLI and curl.
- Browser
- Download file 15.6 kB
-
https://huggingface.co/CopyleftCultivars/agri-vision-detector/resolve/main/docs/HOWTO.md
- Command line
-
hf download hf://CopyleftCultivars/agri-vision-detector/docs/HOWTO.md
-
curl -L -o HOWTO.md https://huggingface.co/CopyleftCultivars/agri-vision-detector/resolve/main/docs/HOWTO.md
15.6 kB
| # HOWTO: reproduce, run, and extend | |
| Everything here was run on one Dell T7610 with Tesla M40s (sm_52), torch 1.13.1, on 2026-10-08. Paths below are the ones used; change `ROOT` as needed. | |
| Scripts live in this repo; bulky data lives under `/mnt/store1/agri` (not in git). | |
| ## 0. Ground rules learned the hard way | |
| - **No Hugging Face transformers `generate()` on old GPUs** (it has crashed this box). Nothing here uses transformers: the detector is a CNN (Ultralytics) and d1 runs in llama.cpp. | |
| - On a shared GPU box, reserve first (`gpusched reserve --gpu 0 ...`), pin to one GPU/NUMA node (`gpu0run`), watch temperature with a guard scoped to **your own PID**, and release when done. | |
| - Never `pkill -f`/`pgrep -f` a pattern that your own shell command contains (it kills your shell). Match on `readlink /proc/PID/exe`, a PID you saved, or a listening port. | |
| - Large downloads: single-stream `curl -L -C -` with retries (HF xet paths failed on this box). Check completeness by size/MD5, not exit code. Re-download if a "CSV" is a few hundred bytes (it was a redirect page: use `curl -L`). | |
| - Disk filled to 98%: download, downscale (max side 640), record in a manifest, delete the raw zip, repeat. | |
| ## 1. Environments | |
| ```bash | |
| python3 -m venv ~/venvs/agri-vision && ~/venvs/agri-vision/bin/pip install boto3 pycocotools pillow numpy requests tqdm # data + analysis, no torch | |
| # detector training: reuse an existing torch 1.13.1+cu117 venv through a .pth file (so pip never touches torch), then add torchvision + ultralytics | |
| python -m venv ~/venvs/agri-yolo && echo "<path-to>/research-pt113/lib/python3.10/site-packages" > ~/venvs/agri-yolo/lib/python3.10/site-packages/zz_pt113.pth | |
| ~/venvs/agri-yolo/bin/pip install "torchvision==0.14.1" --index-url https://download.pytorch.org/whl/cu117 | |
| ~/venvs/agri-yolo/bin/pip install "ultralytics==8.3.0" opencv-python-headless onnx onnxruntime | |
| ``` | |
| Confirm after installing: `pip list | grep -i '^torch '` still shows 1.13.1 in the shared venv. | |
| ## 2. Artemis / Image Safari data (anonymous S3) | |
| ```bash | |
| python s3_list.py ImageSafari/ / 50 # list crops; bucket alliance-artemis-imagesafari, us-west-2, unsigned | |
| python enum_annotations.py # -> annotation_index.json (all COCO JSONs + image dirs per crop) | |
| python fetch_coco.py # downloads box-bearing COCO files (<130 MB each) to /mnt/store1/agri/coco, prints classes | |
| ``` | |
| Facts to know: the real prefix layout differs from the README (`<crop>/annotations/<variety>/standard/<task>/<set>/`); only bean, cowpea and sorghum have true box-detection sets; | |
| several annotation sets are duplicates (Safari bush bean == Artemis bush bean, two identical cowpea JSONs, two overlapping sorghum JSONs). `cowpea_plant` boxes include tiny noisy boxes. | |
| Always draw boxes on a few images (`overlay.py`) before training. | |
| ## 3. Detector | |
| ```bash | |
| python build_dataset.py /mnt/store1/agri/ds_v1 # YOLO-format dataset (3 classes), EXIF-free raw pixels, longest side <=1280 | |
| python resplit.py # per-crop, date/batch-grouped 70/15/15 split (the first greedy split was class-skewed; check class counts per split!) | |
| gpusched reserve --gpu 0 --duration 5h --session <you> --harness claude-code --purpose "YOLO11n detector" | |
| ./run_train.sh yolo11n_v1 60 1.0 640 16 # smoke test first: ./run_train.sh smoke 1 0.05 640 16 (fraction takes the first N files alphabetically) | |
| python eval_detector.py yolo11n_v1 # CPU eval on val+test, torch vs ONNX -> detector_eval_yolo11n_v1.json | |
| ``` | |
| Settings: YOLO11n pretrained, imgsz 640, batch 16, `amp=False` (M40 has no fast fp16), patience 15. ~2.2 min/epoch; early-stopped at epoch 55 (best 40). GPU peaked ~83 C. | |
| `thermal_guard.sh PID GPU LIMIT LOGFILE` kills only that PID after three consecutive reads at/above the limit. | |
| ### Run the exported model | |
| ```bash | |
| python infer_onnx.py best.onnx photo.jpg --conf 0.3 --iou 0.5 --save out.jpg # onnxruntime + numpy + PIL only | |
| ``` | |
| Output tensor `(1, 7, 8400)`: cx, cy, w, h (letterboxed 640 px) + scores for `common_bean, cowpea, sorghum`. For Android use ONNX Runtime Mobile (not tested here). | |
| TFLite: **does not work yet.** `onnx2tf` 1.22.3 + TF 2.15 failed on YOLO11n (detect head, then C2PSA attention) and YOLOv8n (a Conv op). Don't spend long on it without a newer toolchain. | |
| ## 4. Disease / pest data | |
| ```bash | |
| python disease_manifest.py plantdoc|plantwild|plantseg|ip102|cassava # per-source parser: downscale to max 640, append rows to disease_manifest.jsonl | |
| python make_crop_set.py # ~100 Safari field images per crop for crop-ID (excludes probe images by exact key AND by image-hash stem) | |
| python canon.py # prints how raw condition strings map to the 12 coarse concepts | |
| ``` | |
| Sources and exact URLs: PlantDoc `codeload.github.com/pratikkayal/PlantDoc-Dataset/zip/refs/heads/master`; PlantWild `huggingface.co/datasets/uqtwei2/PlantWild/resolve/main/plantwild.zip`; | |
| PlantSeg `zenodo.org/api/records/16665770/files/Plantseg.zip/content` (MD5 23385d877bc84bab51b54630055c75b8); IP102 COCO `zenodo.org/api/records/15546419/files/ip102.zip/content`; | |
| Cassava via HF `Engineer101/cassava-leaf-disease-classification` (individual files; use the repo's file list to find `train_images/part_xxx/`). | |
| **Audit labels by eye** (we did: contact sheets with the mapped label printed on each tile). Findings: PlantWild is noisy; PlantDoc healthy = white-background stock images (style confound). | |
| Leakage checks that mattered: the same photo appeared under several annotation sets (8 training images were probe images; 14 within-set duplicates) — compare image-hash stems, not only keys. | |
| ## 5. d1-omni-600M locally (llama.cpp) and the trunk-feature patch | |
| ```bash | |
| git clone https://github.com/ggml-org/llama.cpp /mnt/store1/agri/llama.cpp-d1 # tested at de7fa0a (2026-10-08); older checkouts lack /v1/systemone | |
| cmake -B build-cuda -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=52 -DLLAMA_CURL=OFF && cmake --build build-cuda -j6 --target llama-server | |
| # models: curl -L -C - the d1-omni-600M-{Q8_0,F16}.gguf and mmproj-d1-omni-600M-{Q8_0,F16}.gguf files from LiquidAI/d1-omni-600M-GGUF | |
| D1_EXPORT_TRUNK=1 D1_DUMP_PATH=/tmp/dump.bin gpu0run ./build-cuda/bin/llama-server -m d1-omni-600M-F16.gguf --mmproj mmproj-d1-omni-600M-F16.gguf \ | |
| --port 8097 -b 4096 -ub 4096 -ngl 99 -np 1 --image-max-tokens 256 | |
| ``` | |
| **The patch** (two small edits, kept outside upstream; `patches/d1-trunk-export.patch`, 17 added lines, verified to apply cleanly to llama.cpp commit de7fa0a): (1) `src/models/lfm2.cpp` — when `D1_EXPORT_TRUNK` is set, `n_embd_out` grows from 3 to 3+1024 and the post-`output_norm` | |
| trunk state at the output rows is concatenated after the 3 score columns; (2) `tools/server/server-context.cpp` — when `D1_DUMP_PATH` is set, append a record `[int32 n_markers, int32 n_embd_out, float32 x n_markers*n_embd_out]` per request. | |
| Requires `-np 1` (records are matched to requests by order). `extract_features.py` asserts the record count/order and that `softmax(dumped logits)` reproduces the server's returned probabilities. | |
| Questions are sent to `POST /v1/systemone` with `files:[data-URL image]` and `questions:{id:{type:choice|score|noul, instructions, criteria}}`. `noul` returns P(true). Throughput on one M40: ~0.45 s/image at 448 px. | |
| CPU-only is ~190 s for a 1080p image: use a GPU. | |
| ### Probe and feature extraction | |
| ```bash | |
| python make_probe_set.py && python sheets.py # 204-image probe + numbered contact sheets (look at them, then fill probe_flags.json) | |
| python probe_run.py http://127.0.0.1:8099 probe_results_f16.jsonl 896 && python probe_analyze.py probe_results_f16.jsonl | |
| python build_extract_lists.py && for n in probe artemis_det safari_crop plantdoc cassava plantwild plantseg ip102; do python style_feats.py $n; done | |
| ./run_extraction.sh # unattended: reserves GPU, starts the patched server under a thermal guard, extracts each list, tears down | |
| ``` | |
| ## 6. Heads | |
| ```bash | |
| python train_head.py --tasks crop,healthy,cond,pest # plain PyTorch on CPU (~2 min); every number next to d1 zero-shot, a style-only control and majority class, with Wilson 95% CIs | |
| python export_heads.py # refit on train+val, save artifacts/heads_v1/*.npz (+json), verify numpy forward == torch | |
| ``` | |
| **Dedup is mandatory.** PlantDoc, PlantWild and PlantSeg contain the same photos (65 of 2,578 PlantDoc images match at cosine >= 0.98 in d1 trunk-feature space; verified by eye). `train_head.py` flags any evaluation image with a >= 0.95 near-duplicate in the training pool (`flagdups`) and drops it from evaluation only. `train_head_v1_before_dedup.py` is the pre-dedup version, kept so the effect is auditable (up to ~4 points). | |
| The saved heads are plain NumPy: `logits = ((x - mu) / sd) @ W.T + b` on the 3,072-d feature vector. | |
| Rules we hold ourselves to: report n and a Wilson interval for every rate; compare with zero-shot and with a no-semantics control; evaluate on a different source/campaign than training | |
| (PlantDoc is the cross-source test; Artemis detection images are the out-of-domain crop test); hand-check flagged samples before quoting a count. | |
| ## 7. Backups | |
| Features, manifests and weights are not "recreatable": they depend on the exact d1 build, patch and hardware. Keep a private copy with a SHA-256 manifest and re-hash the copy. | |
| `artifacts/yolo11n_v1/SHA256SUMS` is an example (copies verified byte-identical). | |
| ## 8. Android (status: builds only) | |
| ```bash | |
| cmake -B build-android -DCMAKE_TOOLCHAIN_FILE=$NDK/build/cmake/android.toolchain.cmake -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-28 -DCMAKE_BUILD_TYPE=Release \ | |
| -DGGML_OPENMP=OFF -DGGML_NATIVE=OFF -DLLAMA_CURL=OFF -DLLAMA_OPENSSL=OFF -DBUILD_SHARED_LIBS=OFF && cmake --build build-android -j3 --target llama-server | |
| ``` | |
| Produced a 185 MB (unstripped) aarch64 ELF with NDK r27c. **Never executed on a device**; latency and memory on a phone are unmeasured. Expect CPU-only (no GPU offload) and test the Q8_0 model (407 MB + 263 MB mmproj). | |
| ## 9. End-to-end demo (CPU only, no GPU needed) | |
| ```bash | |
| # CPU-only build of the patched server (never loads CUDA): cmake -B build-cpu -DGGML_CUDA=OFF -DLLAMA_CURL=OFF && cmake --build build-cpu --target llama-server | |
| D1_EXPORT_TRUNK=1 D1_DUMP_PATH=/tmp/dump.bin ./build-cpu/bin/llama-server -m d1-omni-600M-F16.gguf --mmproj mmproj-d1-omni-600M-F16.gguf --port 8097 -np 1 -t 6 --image-max-tokens 256 & | |
| python diagnose.py photo.jpg --server http://127.0.0.1:8097 --dump /tmp/dump.bin --save out.jpg # detector boxes + crop / healthy / condition head outputs as JSON, with warnings | |
| ``` | |
| ~6-9 s per image on 6 CPU threads (d1 dominates; the ONNX detector is fast). Features from this CPU path matched the stored GPU-extracted features (cosine 1.00000, relative L2 difference ~0.001 on 3 images), so CPU and GPU extraction are interchangeable here. | |
| `diagnose.py` is a research demo: the detector only knows bean/cowpea/sorghum seedlings in Artemis-style photos and will mislabel other crops (it called a cassava leaf "sorghum"). | |
| ## 10. Tiny CNN diagnoser (no d1, no llama.cpp) — added 2026-10-10 | |
| Everything below is plain PyTorch/torchvision/ONNX Runtime (no transformers), run on the same M40 box via the same `gpusched` + `gpu{0,1}run` + scoped thermal-guard protocol as section 0 (`run_cnn.sh GPU NAME args...` does the pinning and guard). | |
| ```bash | |
| # 1. caches (uint8 224x224 squash-resize, row order == meta order; ~8 GB for the disease manifest, ~4 GB CCMT) | |
| python build_img_cache.py 224 # disease_manifest.jsonl minus IP102 -> imgcache/img224.npy, meta.jsonl | |
| python ccmt_manifest.py && python build_cache_generic.py imgcache/ccmt_meta.jsonl imgcache/ccmt 224 # CCMT raw .jpg (see notebook 31: names collide across folders, fetch with uuid suffix) | |
| python ip102full_manifest.py && python build_cache_generic.py imgcache/ip102full_meta.jsonl imgcache/ip102full 224 | |
| # 2. train (MobileNetV3-L from torchvision weights; heads: healthy/diseased + masked 12-way condition); measured on one M40: E1 (21k training images, 14 epochs) ~24 min, E5 (53k images, 14 epochs) ~65 min | |
| ./run_cnn.sh 0 e5_all_ccmt --train plantwild,plantseg,plantdoc,cassava,karaagro,makerere,pretoria,ccmt --epochs 14 [--seed N] [--arch effb0] [--aug strong] | |
| # leave-one-source-out: simply leave a source out of --train; every source not in --train is evaluated as an external test (test split; PlantDoc/CCMT all rows), near-dups excluded | |
| # 3. other tasks | |
| DS=ip102full ./run_cnn_pest.sh 0 pest_ip102full 30 # 97-class pest species, pHash-deduped test | |
| ./run_cnn_crop.sh 0 crop_sites 15 # site-disjoint 5-fold crop ID (Safari) | |
| ./run_cnn_ident.sh 1 species_gbif 15 gbif_amazon taxon group # observer-grouped species ID (GBIF) | |
| # 4. analysis | |
| python calibrate_cnn.py RUN train_srcs eval_srcs # temperature + abstention curves (T fitted on source val only) | |
| python eval_cnn_ccmt.py RUN all # external CCMT evaluation, per class / crop | |
| python eval_tta_ens.py run1,run2,run3 plantdoc,karaagro,ccmt # TTA / seed-ensemble check | |
| python per_condition_eval.py RUN SRC # per-condition error analysis | |
| # 5. export + INT8 (opset 13 needed for per-channel QDQ; PERCENTILE calibration is required - MinMax/Entropy collapse to chance) | |
| OPSET=13 python export_cnn_onnx.py RUN && CALIB=Percentile python quantize_cnn_onnx.py RUN karaagro && python export_cnn_onnx.py RUN # last call: opset-12 fp32 | |
| # 6. run it (onnxruntime + PIL + numpy only) | |
| python infer_tiny.py artifacts/yolo11n_ft1280/best.onnx artifacts/cnn_tiny_v1/cnn_tiny_int8_percentile.onnx photo.jpg --unsure 0.75 --save out.jpg | |
| ``` | |
| Notes: (a) the detector input size is read from the ONNX (1280 for `yolo11n_ft1280`, which must be run at 1280; 960 for v2/v1res960); (b) `infer_tiny.py` runs the classifier on each detected plant crop **and** on the whole image, because the classifier was trained on leaf/close-up and whole-plant photos, not on tiny Safari crops: crops below 24 px are reported as `too_small`; (c) confidences are raw softmax, not calibrated (notebook 33); (d) the trained model has never seen a hemp image and was not evaluated on Amazonian crops for disease. | |
| Detector fine-tune used for the 1280 model: `./run_train.sh yolo11n_ft1280 25 1.0 1280 8 <v1res960/best.pt> 0.002` (GPU0, 3.5 h). Evaluate with `DEV=0 python eval_detector.py yolo11n_ft1280 1280 ft1280` and `DEV=0 python size_recall.py <weights.pt> 1280 ft1280` (model must be a path). | |
| ## 11. Crop ID with web photos (added 2026-10-25) | |
| ```bash | |
| python3 -I gbif_fetch.py /mnt/store1/agri/dl/gbif_crops 150 "Zea mays" "Sorghum bicolor" ... # GBIF/iNaturalist photos | |
| # build imgcache/gbif_crops_meta.jsonl (add a "crop" field per taxon), then: | |
| python build_cache_generic.py imgcache/gbif_crops_meta.jsonl imgcache/gbif_crops 224 | |
| ./run_web_chain.sh # matched web vs control (MAXW=0), seeds 0-2, GPU0, ~25 min each; hold a gpusched reservation first | |
| SEED=0 gpu0run python -I train_crop_final.py crop18_web_final 15 # final all-data model (~7 min) | |
| python -I export_crop18_onnx.py artifacts/crop18_web_v1 # ONNX opset 12 + parity check (CPU) | |
| ``` | |
| Web rows are train-only (fold=-1); the test set is computed from Safari rows so it is identical with and without web photos. | |