Instructions to use CopyleftCultivars/agri-vision-detector with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use CopyleftCultivars/agri-vision-detector with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf CopyleftCultivars/agri-vision-detector:F16 # Run inference directly in the terminal: llama cli -hf CopyleftCultivars/agri-vision-detector:F16
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf CopyleftCultivars/agri-vision-detector:F16 # Run inference directly in the terminal: llama cli -hf CopyleftCultivars/agri-vision-detector:F16
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf CopyleftCultivars/agri-vision-detector:F16 # Run inference directly in the terminal: ./build/bin/llama cli -hf CopyleftCultivars/agri-vision-detector:F16
Use Docker
docker model run hf.co/CopyleftCultivars/agri-vision-detector:F16
- Llama new
- LM Studio
- Jan
- Ollama
How to use CopyleftCultivars/agri-vision-detector with Ollama:
ollama run hf.co/CopyleftCultivars/agri-vision-detector:F16
- Unsloth Desktop
- Docker Model Runner
How to use CopyleftCultivars/agri-vision-detector with Docker Model Runner:
docker model run hf.co/CopyleftCultivars/agri-vision-detector:F16
- Lemonade
How to use CopyleftCultivars/agri-vision-detector with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull CopyleftCultivars/agri-vision-detector:F16
Run and chat with the model
lemonade run user.agri-vision-detector-F16
List all available models
lemonade list
- Atomic Chat
Download docs/HOWTO.md from CopyleftCultivars/agri-vision-detector: direct link, hf CLI and curl.
- Browser
- Download file 15.6 kB
-
https://huggingface.co/CopyleftCultivars/agri-vision-detector/resolve/main/docs/HOWTO.md
- Command line
-
hf download hf://CopyleftCultivars/agri-vision-detector/docs/HOWTO.md
-
curl -L -o HOWTO.md https://huggingface.co/CopyleftCultivars/agri-vision-detector/resolve/main/docs/HOWTO.md
HOWTO: reproduce, run, and extend
Everything here was run on one Dell T7610 with Tesla M40s (sm_52), torch 1.13.1, on 2026-10-08. Paths below are the ones used; change ROOT as needed.
Scripts live in this repo; bulky data lives under /mnt/store1/agri (not in git).
0. Ground rules learned the hard way
- No Hugging Face transformers
generate()on old GPUs (it has crashed this box). Nothing here uses transformers: the detector is a CNN (Ultralytics) and d1 runs in llama.cpp. - On a shared GPU box, reserve first (
gpusched reserve --gpu 0 ...), pin to one GPU/NUMA node (gpu0run), watch temperature with a guard scoped to your own PID, and release when done. - Never
pkill -f/pgrep -fa pattern that your own shell command contains (it kills your shell). Match onreadlink /proc/PID/exe, a PID you saved, or a listening port. - Large downloads: single-stream
curl -L -C -with retries (HF xet paths failed on this box). Check completeness by size/MD5, not exit code. Re-download if a "CSV" is a few hundred bytes (it was a redirect page: usecurl -L). - Disk filled to 98%: download, downscale (max side 640), record in a manifest, delete the raw zip, repeat.
1. Environments
python3 -m venv ~/venvs/agri-vision && ~/venvs/agri-vision/bin/pip install boto3 pycocotools pillow numpy requests tqdm # data + analysis, no torch
# detector training: reuse an existing torch 1.13.1+cu117 venv through a .pth file (so pip never touches torch), then add torchvision + ultralytics
python -m venv ~/venvs/agri-yolo && echo "<path-to>/research-pt113/lib/python3.10/site-packages" > ~/venvs/agri-yolo/lib/python3.10/site-packages/zz_pt113.pth
~/venvs/agri-yolo/bin/pip install "torchvision==0.14.1" --index-url https://download.pytorch.org/whl/cu117
~/venvs/agri-yolo/bin/pip install "ultralytics==8.3.0" opencv-python-headless onnx onnxruntime
Confirm after installing: pip list | grep -i '^torch ' still shows 1.13.1 in the shared venv.
2. Artemis / Image Safari data (anonymous S3)
python s3_list.py ImageSafari/ / 50 # list crops; bucket alliance-artemis-imagesafari, us-west-2, unsigned
python enum_annotations.py # -> annotation_index.json (all COCO JSONs + image dirs per crop)
python fetch_coco.py # downloads box-bearing COCO files (<130 MB each) to /mnt/store1/agri/coco, prints classes
Facts to know: the real prefix layout differs from the README (<crop>/annotations/<variety>/standard/<task>/<set>/); only bean, cowpea and sorghum have true box-detection sets;
several annotation sets are duplicates (Safari bush bean == Artemis bush bean, two identical cowpea JSONs, two overlapping sorghum JSONs). cowpea_plant boxes include tiny noisy boxes.
Always draw boxes on a few images (overlay.py) before training.
3. Detector
python build_dataset.py /mnt/store1/agri/ds_v1 # YOLO-format dataset (3 classes), EXIF-free raw pixels, longest side <=1280
python resplit.py # per-crop, date/batch-grouped 70/15/15 split (the first greedy split was class-skewed; check class counts per split!)
gpusched reserve --gpu 0 --duration 5h --session <you> --harness claude-code --purpose "YOLO11n detector"
./run_train.sh yolo11n_v1 60 1.0 640 16 # smoke test first: ./run_train.sh smoke 1 0.05 640 16 (fraction takes the first N files alphabetically)
python eval_detector.py yolo11n_v1 # CPU eval on val+test, torch vs ONNX -> detector_eval_yolo11n_v1.json
Settings: YOLO11n pretrained, imgsz 640, batch 16, amp=False (M40 has no fast fp16), patience 15. ~2.2 min/epoch; early-stopped at epoch 55 (best 40). GPU peaked ~83 C.
thermal_guard.sh PID GPU LIMIT LOGFILE kills only that PID after three consecutive reads at/above the limit.
Run the exported model
python infer_onnx.py best.onnx photo.jpg --conf 0.3 --iou 0.5 --save out.jpg # onnxruntime + numpy + PIL only
Output tensor (1, 7, 8400): cx, cy, w, h (letterboxed 640 px) + scores for common_bean, cowpea, sorghum. For Android use ONNX Runtime Mobile (not tested here).
TFLite: does not work yet. onnx2tf 1.22.3 + TF 2.15 failed on YOLO11n (detect head, then C2PSA attention) and YOLOv8n (a Conv op). Don't spend long on it without a newer toolchain.
4. Disease / pest data
python disease_manifest.py plantdoc|plantwild|plantseg|ip102|cassava # per-source parser: downscale to max 640, append rows to disease_manifest.jsonl
python make_crop_set.py # ~100 Safari field images per crop for crop-ID (excludes probe images by exact key AND by image-hash stem)
python canon.py # prints how raw condition strings map to the 12 coarse concepts
Sources and exact URLs: PlantDoc codeload.github.com/pratikkayal/PlantDoc-Dataset/zip/refs/heads/master; PlantWild huggingface.co/datasets/uqtwei2/PlantWild/resolve/main/plantwild.zip;
PlantSeg zenodo.org/api/records/16665770/files/Plantseg.zip/content (MD5 23385d877bc84bab51b54630055c75b8); IP102 COCO zenodo.org/api/records/15546419/files/ip102.zip/content;
Cassava via HF Engineer101/cassava-leaf-disease-classification (individual files; use the repo's file list to find train_images/part_xxx/).
Audit labels by eye (we did: contact sheets with the mapped label printed on each tile). Findings: PlantWild is noisy; PlantDoc healthy = white-background stock images (style confound).
Leakage checks that mattered: the same photo appeared under several annotation sets (8 training images were probe images; 14 within-set duplicates) — compare image-hash stems, not only keys.
5. d1-omni-600M locally (llama.cpp) and the trunk-feature patch
git clone https://github.com/ggml-org/llama.cpp /mnt/store1/agri/llama.cpp-d1 # tested at de7fa0a (2026-10-08); older checkouts lack /v1/systemone
cmake -B build-cuda -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=52 -DLLAMA_CURL=OFF && cmake --build build-cuda -j6 --target llama-server
# models: curl -L -C - the d1-omni-600M-{Q8_0,F16}.gguf and mmproj-d1-omni-600M-{Q8_0,F16}.gguf files from LiquidAI/d1-omni-600M-GGUF
D1_EXPORT_TRUNK=1 D1_DUMP_PATH=/tmp/dump.bin gpu0run ./build-cuda/bin/llama-server -m d1-omni-600M-F16.gguf --mmproj mmproj-d1-omni-600M-F16.gguf \
--port 8097 -b 4096 -ub 4096 -ngl 99 -np 1 --image-max-tokens 256
The patch (two small edits, kept outside upstream; patches/d1-trunk-export.patch, 17 added lines, verified to apply cleanly to llama.cpp commit de7fa0a): (1) src/models/lfm2.cpp — when D1_EXPORT_TRUNK is set, n_embd_out grows from 3 to 3+1024 and the post-output_norm
trunk state at the output rows is concatenated after the 3 score columns; (2) tools/server/server-context.cpp — when D1_DUMP_PATH is set, append a record [int32 n_markers, int32 n_embd_out, float32 x n_markers*n_embd_out] per request.
Requires -np 1 (records are matched to requests by order). extract_features.py asserts the record count/order and that softmax(dumped logits) reproduces the server's returned probabilities.
Questions are sent to POST /v1/systemone with files:[data-URL image] and questions:{id:{type:choice|score|noul, instructions, criteria}}. noul returns P(true). Throughput on one M40: ~0.45 s/image at 448 px.
CPU-only is ~190 s for a 1080p image: use a GPU.
Probe and feature extraction
python make_probe_set.py && python sheets.py # 204-image probe + numbered contact sheets (look at them, then fill probe_flags.json)
python probe_run.py http://127.0.0.1:8099 probe_results_f16.jsonl 896 && python probe_analyze.py probe_results_f16.jsonl
python build_extract_lists.py && for n in probe artemis_det safari_crop plantdoc cassava plantwild plantseg ip102; do python style_feats.py $n; done
./run_extraction.sh # unattended: reserves GPU, starts the patched server under a thermal guard, extracts each list, tears down
6. Heads
python train_head.py --tasks crop,healthy,cond,pest # plain PyTorch on CPU (~2 min); every number next to d1 zero-shot, a style-only control and majority class, with Wilson 95% CIs
python export_heads.py # refit on train+val, save artifacts/heads_v1/*.npz (+json), verify numpy forward == torch
Dedup is mandatory. PlantDoc, PlantWild and PlantSeg contain the same photos (65 of 2,578 PlantDoc images match at cosine >= 0.98 in d1 trunk-feature space; verified by eye). train_head.py flags any evaluation image with a >= 0.95 near-duplicate in the training pool (flagdups) and drops it from evaluation only. train_head_v1_before_dedup.py is the pre-dedup version, kept so the effect is auditable (up to ~4 points).
The saved heads are plain NumPy: logits = ((x - mu) / sd) @ W.T + b on the 3,072-d feature vector.
Rules we hold ourselves to: report n and a Wilson interval for every rate; compare with zero-shot and with a no-semantics control; evaluate on a different source/campaign than training (PlantDoc is the cross-source test; Artemis detection images are the out-of-domain crop test); hand-check flagged samples before quoting a count.
7. Backups
Features, manifests and weights are not "recreatable": they depend on the exact d1 build, patch and hardware. Keep a private copy with a SHA-256 manifest and re-hash the copy.
artifacts/yolo11n_v1/SHA256SUMS is an example (copies verified byte-identical).
8. Android (status: builds only)
cmake -B build-android -DCMAKE_TOOLCHAIN_FILE=$NDK/build/cmake/android.toolchain.cmake -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-28 -DCMAKE_BUILD_TYPE=Release \
-DGGML_OPENMP=OFF -DGGML_NATIVE=OFF -DLLAMA_CURL=OFF -DLLAMA_OPENSSL=OFF -DBUILD_SHARED_LIBS=OFF && cmake --build build-android -j3 --target llama-server
Produced a 185 MB (unstripped) aarch64 ELF with NDK r27c. Never executed on a device; latency and memory on a phone are unmeasured. Expect CPU-only (no GPU offload) and test the Q8_0 model (407 MB + 263 MB mmproj).
9. End-to-end demo (CPU only, no GPU needed)
# CPU-only build of the patched server (never loads CUDA): cmake -B build-cpu -DGGML_CUDA=OFF -DLLAMA_CURL=OFF && cmake --build build-cpu --target llama-server
D1_EXPORT_TRUNK=1 D1_DUMP_PATH=/tmp/dump.bin ./build-cpu/bin/llama-server -m d1-omni-600M-F16.gguf --mmproj mmproj-d1-omni-600M-F16.gguf --port 8097 -np 1 -t 6 --image-max-tokens 256 &
python diagnose.py photo.jpg --server http://127.0.0.1:8097 --dump /tmp/dump.bin --save out.jpg # detector boxes + crop / healthy / condition head outputs as JSON, with warnings
~6-9 s per image on 6 CPU threads (d1 dominates; the ONNX detector is fast). Features from this CPU path matched the stored GPU-extracted features (cosine 1.00000, relative L2 difference ~0.001 on 3 images), so CPU and GPU extraction are interchangeable here.
diagnose.py is a research demo: the detector only knows bean/cowpea/sorghum seedlings in Artemis-style photos and will mislabel other crops (it called a cassava leaf "sorghum").
10. Tiny CNN diagnoser (no d1, no llama.cpp) — added 2026-10-10
Everything below is plain PyTorch/torchvision/ONNX Runtime (no transformers), run on the same M40 box via the same gpusched + gpu{0,1}run + scoped thermal-guard protocol as section 0 (run_cnn.sh GPU NAME args... does the pinning and guard).
# 1. caches (uint8 224x224 squash-resize, row order == meta order; ~8 GB for the disease manifest, ~4 GB CCMT)
python build_img_cache.py 224 # disease_manifest.jsonl minus IP102 -> imgcache/img224.npy, meta.jsonl
python ccmt_manifest.py && python build_cache_generic.py imgcache/ccmt_meta.jsonl imgcache/ccmt 224 # CCMT raw .jpg (see notebook 31: names collide across folders, fetch with uuid suffix)
python ip102full_manifest.py && python build_cache_generic.py imgcache/ip102full_meta.jsonl imgcache/ip102full 224
# 2. train (MobileNetV3-L from torchvision weights; heads: healthy/diseased + masked 12-way condition); measured on one M40: E1 (21k training images, 14 epochs) ~24 min, E5 (53k images, 14 epochs) ~65 min
./run_cnn.sh 0 e5_all_ccmt --train plantwild,plantseg,plantdoc,cassava,karaagro,makerere,pretoria,ccmt --epochs 14 [--seed N] [--arch effb0] [--aug strong]
# leave-one-source-out: simply leave a source out of --train; every source not in --train is evaluated as an external test (test split; PlantDoc/CCMT all rows), near-dups excluded
# 3. other tasks
DS=ip102full ./run_cnn_pest.sh 0 pest_ip102full 30 # 97-class pest species, pHash-deduped test
./run_cnn_crop.sh 0 crop_sites 15 # site-disjoint 5-fold crop ID (Safari)
./run_cnn_ident.sh 1 species_gbif 15 gbif_amazon taxon group # observer-grouped species ID (GBIF)
# 4. analysis
python calibrate_cnn.py RUN train_srcs eval_srcs # temperature + abstention curves (T fitted on source val only)
python eval_cnn_ccmt.py RUN all # external CCMT evaluation, per class / crop
python eval_tta_ens.py run1,run2,run3 plantdoc,karaagro,ccmt # TTA / seed-ensemble check
python per_condition_eval.py RUN SRC # per-condition error analysis
# 5. export + INT8 (opset 13 needed for per-channel QDQ; PERCENTILE calibration is required - MinMax/Entropy collapse to chance)
OPSET=13 python export_cnn_onnx.py RUN && CALIB=Percentile python quantize_cnn_onnx.py RUN karaagro && python export_cnn_onnx.py RUN # last call: opset-12 fp32
# 6. run it (onnxruntime + PIL + numpy only)
python infer_tiny.py artifacts/yolo11n_ft1280/best.onnx artifacts/cnn_tiny_v1/cnn_tiny_int8_percentile.onnx photo.jpg --unsure 0.75 --save out.jpg
Notes: (a) the detector input size is read from the ONNX (1280 for yolo11n_ft1280, which must be run at 1280; 960 for v2/v1res960); (b) infer_tiny.py runs the classifier on each detected plant crop and on the whole image, because the classifier was trained on leaf/close-up and whole-plant photos, not on tiny Safari crops: crops below 24 px are reported as too_small; (c) confidences are raw softmax, not calibrated (notebook 33); (d) the trained model has never seen a hemp image and was not evaluated on Amazonian crops for disease.
Detector fine-tune used for the 1280 model: ./run_train.sh yolo11n_ft1280 25 1.0 1280 8 <v1res960/best.pt> 0.002 (GPU0, 3.5 h). Evaluate with DEV=0 python eval_detector.py yolo11n_ft1280 1280 ft1280 and DEV=0 python size_recall.py <weights.pt> 1280 ft1280 (model must be a path).
11. Crop ID with web photos (added 2026-10-25)
python3 -I gbif_fetch.py /mnt/store1/agri/dl/gbif_crops 150 "Zea mays" "Sorghum bicolor" ... # GBIF/iNaturalist photos
# build imgcache/gbif_crops_meta.jsonl (add a "crop" field per taxon), then:
python build_cache_generic.py imgcache/gbif_crops_meta.jsonl imgcache/gbif_crops 224
./run_web_chain.sh # matched web vs control (MAXW=0), seeds 0-2, GPU0, ~25 min each; hold a gpusched reservation first
SEED=0 gpu0run python -I train_crop_final.py crop18_web_final 15 # final all-data model (~7 min)
python -I export_crop18_onnx.py artifacts/crop18_web_v1 # ONNX opset 12 + parity check (CPU)
Web rows are train-only (fold=-1); the test set is computed from Safari rows so it is identical with and without web photos.