Haider3D/plantdx-models

Model Card: PlantDx leaf disease classifier

Status: v1, 23 Oct 2026. Follows Mitchell et al. (2019), "Model Cards for Model Reporting". Every number links to a file in results/ (committed snapshot, regenerated by scripts/snapshot_results.py) or report/tables/, which in turn come from ClearML tasks (offline session ids in results/offline_sessions.jsonl, imported to the ClearML server on 25 Oct 2026; mapping in results/imported_sessions.json). Draft for Haider Ali to review.

Field	Value
Project	PlantDx, 5CCSAGAP Small Group Project, King's College London
Team	Haider Ali (solo team). Team number: TBC
Repository	https://github.com/HaiderAli3D/plantdx-5ccsagap
Model weights	https://huggingface.co/Haider3D/plantdx-models (folders cnn/, resnet18/, mobilenetv3/)
Live app	https://huggingface.co/spaces/Haider3D/plantdx
Related docs	docs/data.md, docs/requirements.md, docs/decisions.md (D-020 to D-026), docs/profiling.md

1. Model details

Item	Value
Task	Single-label image classification: one leaf photo in, one of 39 labels out (38 crop and condition classes over 14 crops, plus Background_without_leaves)
Primary model	From-scratch VGG-style CNN (src/plantdx/models/cnn.py): 4 blocks of [Conv3x3, BatchNorm, ReLU] x 2 then MaxPool, channels 32, 64, 128, 256, global average pooling, dense 256 with ReLU, linear 39. No dropout. 1,249,031 parameters, 1.69 GFLOPs per 128x128 image, 4.8 MB weights
Training	Full training split (38,813 images), 128 px, moderate augmentation, Adam lr 1e-3, batch 64, cosine schedule over 40 epochs, early stopping on validation accuracy (patience 3, from epoch 20), bf16 mixed precision on an RTX 4090 Laptop GPU. Run cnn_full_aug_min20, best epoch 39, seed 42
How the configuration was chosen	30-trial Optuna search on a 25% subset (report/tables/hpo_trials.csv), then full-data variants compared on validation accuracy and robustness (D-025, report/tables/model_selection.csv)
Comparison models (not the primary)	ImageNet-pretrained ResNet18 and MobileNetV3-Large, fine-tuned at 224 px; ResNet18 with a frozen backbone. Offered in the app as labelled comparisons
Input	RGB image of any size: shorter side resized to 128, centre-cropped to 128x128, normalised with training-split mean and std
Output	Softmax over 39 classes; the app shows the top 5, a Grad-CAM heatmap and warnings
Low-confidence threshold	0.83, chosen on the validation split as the lowest top-1 confidence at which shown predictions are 99.9% accurate (report/tables/threshold_cnn_full_aug_min20.json)
Framework	PyTorch 2.14.1, torchvision 0.29.1, Python 3.12
Licence	Code MIT (as pyproject.toml). Weights released under the same terms; the training data is CC0 (see docs/data.md)

2. Intended use

• Primary use: an educational demonstrator of a complete ML workflow, and a first-pass hint about a symptom on a single, well-lit leaf photographed against a plain background, for one of the 14 supported crops.
• Users: module markers and students; hobby growers who want a hint before asking an expert; agronomists or extension workers only as a teaching aid.
• Required disclaimer (shown in the app): PlantDx is a learning and decision-support tool, not agronomic or professional advice. It was trained on lab photos of single leaves and has not been validated on field photos. Consult an agronomist or extension worker before acting, especially before using pesticides.

3. Out-of-scope use

• Field photos. Measured: on 2,578 real-world PlantDoc photos the model is right only 9.7% of the time (Section 6). Do not use it on photos of plants in the field, several leaves, fruit or whole plants.
• Any treatment, pesticide, quarantine, crop-destruction, insurance or trading decision without an expert.
• Crops or diseases outside the 39 labels: the model will still pick one of its labels.
• Severity estimation, lesion localisation or several diseases on one leaf.
• Any use involving people or personal data.

4. Factors

• Crop and class: 14 crops with very different numbers of classes (Tomato 10, five crops 1) and a 36x class imbalance.
• Image quality: blur, brightness, contrast, rotation, compression and resolution are tested (Section 5.3).
• Capture conditions: all training images are lab photos of one leaf on a plain background. Field conditions are the main untested factor and are measured separately on PlantDoc (Section 6).
• Not represented: geography, cultivar, growth stage, camera and season are not recorded in the data, so performance across them is unknown.

5. Quantitative analysis (held-out test split, 8,308 images, evaluated once)

Source: results/eval/eval_cnn_full_aug_min20/report.json and tables/.

5.1 Headline metrics against the requirements

Requirement	Target	Result	Met
R1 macro F1	at least 0.95	0.9950	Yes
R1 lowest class recall	at least 0.85	0.948 (Corn: Cercospora leaf spot)	Yes
R2 top-5 accuracy	at least 0.99	1.000	Yes
R3 worst accuracy drop over seven perturbations at moderate severity	at most 5 points for each	3.0 points (contrast x0.6)	Yes
R5 CPU latency, 2 threads	median under 1 s	11 ms median, 15 ms p95	Yes
R6 background class	detected	precision 1.000, recall 1.000	Yes
R6 calibration	report ECE	ECE 0.0033	Reported
R7 same config and seed reproduces test accuracy	within 0.5 points	0.01 points (rerun cnn_full_aug_full40_seed42)	Yes

Overall test accuracy is 99.64% and weighted F1 0.9964. Across three training seeds trained for the full schedule, validation accuracy spread was 0.25 points; with early stopping able to fire on noisy epochs it was 0.79 points (D-024, D-025).

5.2 By crop (fairness proxy)

Crop	Classes	Test images	Macro F1	Lowest class recall
Corn	4	578	0.983	0.948
Potato	3	323	0.989	0.957
Blueberry	1	225	0.991	1.000
Apple	4	475	0.993	0.979
Pepper (bell)	2	370	0.994	0.987
Tomato	10	2,721	0.996	0.987
Peach, Grape, Soybean, Orange	1 to 4	2,595	0.999	0.994
Background, Cherry, Raspberry, Squash, Strawberry	1 to 2	1,021	1.000	1.000

The weakest crop is Corn, where Cercospora (grey) leaf spot and Northern leaf blight are confused in both directions (3 test images each way). The smallest class, Potato healthy (23 test images), has recall 0.957. Blueberry healthy attracts a few Apple leaves. Differences between crops are small on lab data; they matter far more on field photos (Section 6).

5.3 Robustness (stratified 1,952-image test subset, 7 perturbations x 3 severities)

Worst drop at moderate severity over all seven perturbations (R3): 3.0 points, for contrast x0.6; blur, brightness and rotation each cost at most 0.9 points. The no-augmentation ablation, otherwise as accurate on clean images, drops up to 37.3 points, so augmentation is what makes the model robust. At the strongest severity JPEG quality 10 still costs 33.7 points. Full table: tables/robustness.csv; figure: report/figures/robustness_comparison.png.

5.4 Comparison models (same test split)

Model	Test acc	Macro F1	CPU p50	Size
CNN (primary)	0.9964	0.9950	11 ms	4.8 MB
ResNet18 fine-tuned	0.9989	0.9985	24 ms	42.8 MB
MobileNetV3-L fine-tuned	0.9990	0.9981	10 ms	16.3 MB
ResNet18 frozen backbone	0.9491	0.9396	24 ms	42.8 MB

6. Lab-to-field gap (out-of-distribution check)

All 2,578 PlantDoc photos (Singh et al., 2020; CC BY 4.0; geraldmc/plantdoc-full at revision bd2f049c) map to a PlantVillage class and none were used in training (report/tables/plantdoc.csv).

Model	Top-1	Top-5	Right crop	Called "no leaf"	Warning shown	Mean confidence when wrong
CNN (primary)	9.7%	32.0%	23.1%	29.5%	51.2%	77.1%
ResNet18 fine-tuned	14.3%	42.2%	26.5%	37.6%	14.9%	76.5%
MobileNetV3-L fine-tuned	15.6%	45.9%	27.3%	37.7%	39.3%	80.5%

On field photos the model is close to useless. The safeguards help only partly: about half of the field photos trigger the low-confidence warning and almost 30% are labelled "no leaf", but the remaining predictions are still wrong about 90% of the time, often with high confidence. ImageNet pretraining helps a little but does not close the gap. This is the most important limitation of PlantDx and is stated in the app.

7. Training and evaluation data

• Training: DScomp380/plant_village at revision 5ce680f8 (55,447 images, CC0), the Mendeley "without augmentation" release of PlantVillage. Group-aware stratified split 70/15/15 (seed 42) that keeps 677 near-duplicate groups within one split; a plain split would have leaked 530 duplicate pairs (D-020). See docs/data.md.
• Evaluation: the held-out test split (8,308 images), and PlantDoc for field photos.

8. Ethical considerations and Responsible AI

• Human oversight. Every result carries the disclaimer; diseased predictions tell the user to confirm with an expert; low-confidence and no-leaf results are shown as warnings with advice to retake the photo; users can report a wrong prediction.
• Transparency. Top-5 probabilities are always shown, and a Grad-CAM heatmap shows which part of the image drove the prediction. Calibration on lab data is good (ECE 0.0033), but confidence is not trustworthy on field photos (Section 6).
• Privacy. Photos are used only for the prediction. Gradio keeps a temporary copy of each upload and heatmap on the server's disk, which the app deletes within about 40 minutes (delete_cache=(600, 1800)). A photo is saved only if the user ticks a consent box and reports a prediction, under a server-generated file name with the model, prediction and user-supplied label; no other data is collected. Space storage is not persistent, so reports may be lost on restart.
• Security. No secrets in the repository or Space; dependencies pinned; the flagging endpoint only stores processed images, never client-supplied paths (fixed after a security review).
• Bias. Crops with one class (Soybean, Blueberry, Raspberry only healthy; Orange, Squash only diseased) cannot distinguish healthy from diseased leaves of that crop at all: a diseased Soybean leaf will be called healthy or another crop. Geographic and cultivar coverage are unknown.
• Misuse and failure modes. Over-trust of a confident but wrong answer on a field photo is the main risk; mitigated by the disclaimer, warnings, the About tab and this card, but not eliminated.

9. Environmental impact

Measured with codecarbon (OfflineEmissionsTracker, UK grid) for every training run: 48 metered runs (search trials, baselines, final models, seed repeats, ablations), 4.12 GPU hours, 0.76 kWh, 0.181 kg CO2e. CPU power on this laptop is estimated from its TDP because RAPL is not available, and evaluation runs were not metered, so these are lower bounds. Choices that kept the cost down: prototyping on a 25% subset, median pruning that stopped 14 of 30 search trials early, early stopping, reusing results when the search repeated a configuration, a small model (1.25M parameters) and CPU-friendly 128 px inference.

10. Caveats and recommendations

• Retrain or fine-tune on field photos (for example PlantDoc's training split) before any real-world use, and re-measure on a held-out field set.
• Add an explicit out-of-distribution detector; the background class and the confidence threshold do not catch most field photos.
• Collect metadata (location, cultivar, camera) to make fairness across growers measurable.