Object Detection
Safetensors
stomata
plant-phenotyping
microscopy
benchmark

StomataBench: Measuring Taxonomic Generalization in Stomatal Detection

Sainath Reddy Gummi, Hankui K. Zhang, Chulwoo Pack, Shyam Solanki, Young Chang

This repository holds the 15 fine-tuned stomatal detectors of the paper. Each was fine-tuned once on OD-Train with a single stomata class.

What this repository holds

One folder per detector. The folder names are the model IDs of the paper, with x in place of ×.

Model ID Folder Weight file Config file(s) Framework Licence
FRCNN-R50-1× FRCNN-R50-1x/ StomataBench-OD_FASTER-R-CNN-R50-FPN-1x.pth config3.yaml Detectron2 CC-BY-SA-3.0 (weights), Apache-2.0 (code)
FRCNN-R50-3× FRCNN-R50-3x/ StomataBench-OD_FASTER-R-CNN-R50-FPN-3x.pth config3.yaml Detectron2 CC-BY-SA-3.0 (weights), Apache-2.0 (code)
FRCNN-R101-1× FRCNN-R101-1x/ StomataBench-OD_FASTER-R-CNN-R101-FPN-1x.pth config3.yaml Detectron2 CC-BY-SA-3.0 (weights), Apache-2.0 (code)
FRCNN-X101 FRCNN-X101/ StomataBench-OD_FRCNN-X101.pth config.yaml Detectron2 CC-BY-SA-3.0 (weights), Apache-2.0 (code)
Retina-R50-1× Retina-R50-1x/ StomataBench-OD_Retina-R50-1x.pth config5.yaml Detectron2 CC-BY-SA-3.0 (weights), Apache-2.0 (code)
Retina-R50-3× Retina-R50-3x/ StomataBench-OD_RETINA-NET-RESNET50-FPN-3x.pth config5.yaml Detectron2 CC-BY-SA-3.0 (weights), Apache-2.0 (code)
Retina-R101-3× Retina-R101-3x/ StomataBench-OD_RETINA-NET-RESNET101-FPN-3x.pth config5.yaml Detectron2 CC-BY-SA-3.0 (weights), Apache-2.0 (code)
YOLOv8 YOLOv8/ StomataBench-OD_YOLOv8.pt args.yaml Ultralytics AGPL-3.0
YOLOv11 YOLOv11/ StomataBench-OD_YOLOv11.pt args.yaml Ultralytics AGPL-3.0
YOLOv12 YOLOv12/ StomataBench-OD_YOLOv12.pt args.yaml Ultralytics AGPL-3.0
YOLOv26 YOLOv26/ StomataBench-OD_YOLOv26.pt args.yaml Ultralytics AGPL-3.0
RT-DETRv2 RT-DETRv2/ model.safetensors config.json, preprocessor_config.json HuggingFace Transformers Apache-2.0
RT-DETRv4 RT-DETRv4/ StomataBench-OD_RT-DETRv4.pth none Official RT-DETRv4 code Apache-2.0 (DINOv3 teacher: see Licences)
GDINO-SB GDINO-SB/ model.safetensors config.json, preprocessor_config.json HuggingFace Transformers Apache-2.0
SAM3-OD SAM3-OD/ StomataBench-OD_SAM3.pt config.yaml, config_resolved.yaml Official SAM 3 code SAM License

The RT-DETRv2/ folder also holds the HuggingFace Trainer state files optimizer.pt, rng_state.pth, scaler.pt, scheduler.pt, trainer_state.json, training_args.bin. They are not needed for inference. The RT-DETRv4/ folder holds the checkpoint only.

This card reports no results. In the paper, the detectors are evaluated unchanged on OD-Valid and on four held-out test sets (T-MAWP, T-DCWP, T-MSWP, T-GSH), with COCO localization metrics and with count and density measures. The results are in the paper and in the dataset repository.

Loading

Minimal examples. The scripts used in the paper are in the code repository.

from huggingface_hub import hf_hub_download

repo = "Sainath001/stomatabench_neurips_weights"

# Ultralytics (YOLOv8, YOLOv11, YOLOv12, YOLOv26)
from ultralytics import YOLO
yolo = YOLO(hf_hub_download(repo, "YOLOv8/StomataBench-OD_YOLOv8.pt"))

# Detectron2 (Faster R-CNN and RetinaNet folders)
from detectron2.config import get_cfg
from detectron2.engine import DefaultPredictor
cfg = get_cfg()
cfg.merge_from_file(hf_hub_download(repo, "FRCNN-R50-1x/config3.yaml"))
cfg.MODEL.WEIGHTS = hf_hub_download(repo, "FRCNN-R50-1x/StomataBench-OD_FASTER-R-CNN-R50-FPN-1x.pth")
predictor = DefaultPredictor(cfg)

# HuggingFace Transformers (RT-DETRv2)
from transformers import AutoImageProcessor, AutoModelForObjectDetection
processor = AutoImageProcessor.from_pretrained(repo, subfolder="RT-DETRv2")
rtdetr = AutoModelForObjectDetection.from_pretrained(repo, subfolder="RT-DETRv2")

GDINO-SB loads with AutoModelForZeroShotObjectDetection.from_pretrained(repo, subfolder="GDINO-SB"). Its folder has no tokenizer files, so the processor comes from IDEA-Research/grounding-dino-base (AutoProcessor.from_pretrained("IDEA-Research/grounding-dino-base")), as recorded in the run record of every GDINO-SB prediction file (processor_id). RT-DETRv4 and SAM3-OD need the official RT-DETRv4 and SAM 3 code. The inference scripts and the settings used in the paper are in the code repository (inference/ and scripts/run_inference_*.sh).

Model families

The detectors span five architecture families. The descriptions below follow the paper's Related Work section.

  • Two-stage convolutional detectors (Faster R-CNN [1, 2, 3]) generate class-agnostic region proposals with a Region Proposal Network and then refine these candidates using a second-stage classifier and box regressor. This proposal-refinement design remains a strong baseline for small-object detection because candidate regions are explicitly revisited before final prediction.
  • One-stage convolutional detectors (RetinaNet [2, 3, 4]) remove the proposal stage and directly predict boxes and class probabilities over multi-scale feature maps, using focal loss to address foreground–background imbalance.
  • YOLO-family detectors [5] represent a deployment-oriented branch of one-stage detection, emphasizing fast single-pass inference, efficient feature aggregation, and strong training-time augmentation. YOLO26-X further reflects the recent shift toward end-to-end, NMS-free real-time detection.
  • Real-time deformable transformer detectors cast detection as set prediction. RT-DETRv2 and RT-DETRv4 [6, 7] extend this paradigm toward real-time inference through efficient multi-scale encoding and a deformable-attention decoder. RT-DETRv4 further incorporates training-time vision-foundation-model distillation, using DINOv3 as a teacher through Deep Semantic Injection and Gradient-guided Adaptive Modulation, while preserving inference-time efficiency [7, 8].
  • Foundation-model-derived detectors recast category-specific detection as language-conditioned or concept-prompt-conditioned set prediction. Grounding DINO with a Swin-Base backbone (GDINO-SB) [9] represents language-aligned open-set detection. It combines a visual backbone with a BERT-based text encoder, fuses image and text features through cross-modal attention, and uses language-guided query selection followed by a cross-modality decoder to produce bounding boxes grounded to the input phrase. SAM 3 [10] is a promptable-concept vision foundation model. Unlike earlier SAM models, it includes an image-level detector that can localize all instances matching a concept prompt. It combines a vision encoder, text encoder, geometry encoder, DETR-style encoder–decoder, learned object queries, and a mask decoder. SAM3-OD denotes the paper's detection-only use of the official SAM 3 model rather than a modified architecture.

Including GDINO-SB and SAM3-OD lets StomataBench compare foundation-model-derived detectors with closed-set detectors under the same shifts, without isolating which of their components explains a difference.

Training

Fifteen open-source object detectors were evaluated, spanning five architecture families: two-stage convolutional detectors, one-stage convolutional detectors, YOLO-family detectors, real-time deformable transformer detectors, and foundation-model-derived detectors. The benchmark includes Faster R-CNN variants with ResNet and ResNeXt backbones, RetinaNet, YOLOv8/11/12/26, RT-DETRv2, RT-DETRv4, GDINO-SB, and SAM3-OD [1, 2, 3, 4, 5, 6, 7, 9, 10, 11]. Model sizes range from 31.4M to 839.4M parameters. All detectors were fine-tuned once on OD-Train using a single stomata class and evaluated unchanged on every held-out test set. No model was given taxonomic labels, dataset identity, or test-set-specific tuning. Architectures, parameter counts, learning rates, training schedules, and batch sizes are summarized in the training-settings table below (Table S7 of the paper).

Pre-processing and training. Each detector used its native framework pre-processing pipeline to avoid introducing model-specific custom data handling. Detectron2 models were trained from COCO-format annotations and used the standard shortest-side resize of 800 px with maximum size 1333 px and ImageNet BGR normalization [3]. Ultralytics YOLO models used YOLO-format annotations, letterboxed inputs to 1024×1024, and retained the default Ultralytics training augmentations [5]. RT-DETRv2 used the HuggingFace auto-processor with images passed at native 1280×1024 resolution and ImageNet RGB normalization [6]. RT-DETRv4 used its official training configuration with 640×640 RGB inputs [7, 8]. GDINO-SB used the official HuggingFace processor with the text prompt "a stomata ." [9, 12]. SAM3-OD used the ODinW text-only configuration with 1008×1008 inputs and the prompt "stomata" during fine-tuning and inference [10]. All models were optimized with AdamW. All pretrained checkpoints were fine-tuned end-to-end on OD-Train.

Training settings of the detectors, all fine-tuned on OD-Train with a single stomata class. Params: total parameters in millions. LR: peak learning rate. Schedule: total training budget. BS: batch size per step. Every GPU has 80 GB.

Model ID Architecture Params (M) LR Schedule BS Train GPU
Two-stage convolutional detectors [1, 3]
FRCNN-R50-1× Faster R-CNN, ResNet-50-FPN, 1× 41.4 5×10-5 25,000 iters 4 1×A100
FRCNN-R50-3× Faster R-CNN, ResNet-50-FPN, 3× 41.4 5×10-5 25,000 iters 4 1×A100
FRCNN-R101-1× Faster R-CNN, ResNet-101-FPN, 1× 60.4 5×10-5 25,000 iters 4 1×A100
FRCNN-X101 Faster R-CNN, ResNeXt-101-FPN, 3× 104.8 5×10-5 25,000 iters 4 1×A100
One-stage convolutional detectors [3, 4]
Retina-R50-1× RetinaNet, ResNet-50-FPN, 1× 38.0 5×10-5 25,000 iters 4 1×A100
Retina-R50-3× RetinaNet, ResNet-50-FPN, 3× 38.0 5×10-5 25,000 iters 4 1×A100
Retina-R101-3× RetinaNet, ResNet-101-FPN, 3× 57.1 5×10-5 25,000 iters 4 1×A100
YOLO-family detectors [5, 11]
YOLOv8 YOLOv8-X 68.2 8.7×10-6 400 epochs 14 2×A100
YOLOv11 YOLOv11-X 56.9 8.7×10-6 400 epochs 14 2×A100
YOLOv12 YOLOv12-X 59.1 8.7×10-6 400 epochs 14 2×A100
YOLOv26 YOLOv26-X 58.8 8.7×10-6 400 epochs 14 2×A100
Transformer detectors: RT-DETRv2 [6], RT-DETRv4 [7, 8]
RT-DETRv2 RT-DETRv2, ResNet-50 42.9 5×10-5 50 epochs 8 1×A100
RT-DETRv4 RT-DETRv4-L, HGNetV2-B4, DINOv3 teacher 31.4 2.5×10-4 30 epochs 16 1×H100
Foundation-model-derived detectors: GDINO-SB [9], SAM3-OD [10]
GDINO-SB Grounding-DINO, Swin-Base 232.8 2×10-5 50 epochs 4 1×A100
SAM3-OD SAM3, ODinW text-only configuration 839.4 8×10-6 60 epochs 32 2×H100

Training data: OD-Train

All 15 detectors were fine-tuned on OD-Train only.

The training set consists of 3,154 microscopy images and 134,802 annotated stomatal complexes across 21 species in 10 botanical families (Table S1 and Figure S1 of the paper, reproduced below). Coverage is concentrated in Rosales, with secondary representation from Ericales, Cornales, Ranunculales, Vitales, Fagales, Sapindales, Dipsacales, and Laurales. Each stomatal complex is annotated as a single-class (stomata) axis-aligned bounding box.

Imaging and annotation protocol. OD-Train images were sampled from living accessions at the Arnold Arboretum of Harvard University, Boston. All images were captured by the authors using a ProScope HR handheld digital microscope (USB 2.0, 400× magnification) connected to a tripod-mounted laptop running ProScope HR v1.2.1. Image resolution is 1280×1024 px, with calibration scale s = 0.5988 µm px-1 (167 px per 0.1 mm) measured using a stage micrometer. Imaging was performed on the abaxial (lower) leaf surface. The clear contact dome was gently pressed against the leaf to stabilize the field of view, and the integrated LED ring provided coaxial fill illumination. Field sessions were conducted under natural daylight, and images with strong specular highlights from direct solar reflections were excluded during annotation. For hardwood species, the lowest accessible branch was sampled. Visible stomata were annotated as bounding boxes in RectLabel Pro under the supervision of a botanist. No image augmentation was applied before annotation.

OD-Train species composition (21 species across 10 families). Mean density per species ranges from 30.92 to 277.27 stomata mm-2.

Accession ID(s) Scientific name Family Order Images (N) Stomata (N) GT density (mm-2)
403-97*C & 905-85*A Actinidia arguta Actinidiaceae Ericales 161 4,662 61.61
257-2006*B Actinidia macrosperma Actinidiaceae Ericales 145 5,110 74.99
107-2004*B Calycanthus 'Venus' Calycanthaceae Laurales 55 3,165 122.44
1690-77*B Clematis terniflora Ranunculaceae Ranunculales 308 6,900 47.67
276*A Cotinus coggygria Anacardiaceae Sapindales 56 5,285 200.81
14714-1*A Hydrangea paniculata 'Praecox' Hydrangeaceae Cornales 434 6,306 30.92
427-2016*A Malus 'Blanche Ames' Rosaceae Rosales 134 10,609 168.46
1951-80*A, 1951-80*B, 210-2001*A, 608-2016*A & 12198*B Malus hupehensis Rosaceae Rosales 240 12,063 106.95
28-2013*A Malus ioensis Rosaceae Rosales 49 4,966 215.64
79-2009*B, 51-2009*A, 78-2009*B, 48-2009*A, 49-2009*A, 79-2009*A, 75-2009*A & 74-2009*B Malus sieversii Rosaceae Rosales 204 12,591 131.33
49-98*A, 46-98*A, 228-2013*B & 228-2013*C Malus spontanea Rosaceae Rosales 128 6,724 111.77
80-2017C, 47-98*B & 50-2011B Malus toringoides Rosaceae Rosales 62 4,685 160.78
510-94*A Malus × zumi Rosaceae Rosales 172 7,846 97.06
596-86*A Prinsepia sinensis Rosaceae Rosales 226 6,552 61.69
119-96*A Prunus 'Washi-no-o' Rosaceae Rosales 75 3,712 105.31
13-2016*A Quercus rubra Fagaceae Fagales 116 15,116 277.27
80-2012*A Rosa 'Queen Elizabeth' Rosaceae Rosales 63 1,533 51.78
713-2010*MASS Rosa 'Seafoam' Rosaceae Rosales 162 7,513 98.68
531-34*C Stewartia gemmata Theaceae Ericales 54 1,308 51.54
158-2004*A Viburnum sargentii 'Onondaga' Adoxaceae Dipsacales 55 1,029 39.81
332-84*A Vitis vinifera × amurensis 'Beihong' Vitaceae Vitales 255 7,127 59.47
Total (21 species, 10 families) 3,154 134,802

OD-Train micrographs

Representative stomata micrographs of diverse woody plants from OD-Train (300 µm scale bar).

Intended use and limitations

  • The checkpoints are released for research use, to reproduce the paper and to compare stomatal detectors under distribution shift.
  • Benchmark performance is not a deployment guarantee. Target-domain calibration, confidence selection on independent validation data, and human review remain necessary.
  • The detectors predict one class, stomata, as axis-aligned bounding boxes of stomatal complexes. They do not distinguish guard cells from pavement cells, annotate pores, estimate aperture state, or measure pore-level geometry.
  • OD-Train is taxonomically broad within woody eudicots (plus one magnoliid) but does not include monocots, gymnosperms, ferns, or basal angiosperms (the ANA grade).
  • OD-Train images come from one microscope and one annotation protocol. They were collected without controlled CO₂, humidity, water status, light history, developmental stage, or time-of-day metadata.

Licences

Each fine-tuned checkpoint inherits the licence of the upstream code and pretrained weights it was built from. The licences below were checked against the official sources listed. Redistribution must follow the terms of each upstream licence.

Checkpoints Licence of the fine-tuned weights Official sources
FRCNN-R50-1x, FRCNN-R50-3x, FRCNN-R101-1x, FRCNN-X101, Retina-R50-1x, Retina-R50-3x, Retina-R101-3x CC-BY-SA-3.0 (pretrained weights from the Detectron2 Model Zoo). Detectron2 code is Apache-2.0. Detectron2 LICENSE, Detectron2 Model Zoo, License section
YOLOv8, YOLOv11, YOLOv26 AGPL-3.0 (Ultralytics Enterprise License as the alternative) Ultralytics LICENSE, Ultralytics licensing, YOLO26 docs
YOLOv12 AGPL-3.0 Official YOLOv12 repository, Ultralytics LICENSE
RT-DETRv2 Apache-2.0 Official RT-DETR / RT-DETRv2 repository, HuggingFace Transformers LICENSE, PekingU/rtdetr_v2_r50vd
RT-DETRv4 Apache-2.0. DINOv3 teacher: DINOv3 License, see the note below. Official RT-DETRv4 repository, DINOv3 License
GDINO-SB Apache-2.0 IDEA-Research/grounding-dino-base, Official Grounding DINO repository
SAM3-OD SAM License SAM 3 LICENSE, facebook/sam3

Notes on the table:

  • Detectron2. Each Detectron2 config in this repository records MODEL.WEIGHTS as an ImageNet-pretrained backbone from the Detectron2 Model Zoo, for example detectron2://ImageNetPretrained/MSRA/R-50.pkl. The Model Zoo states that all models available for download through it are licensed under CC-BY-SA-3.0.
  • Ultralytics. Ultralytics states that models trained with its code fall under AGPL-3.0 by default. YOLOv12 was trained with Ultralytics from yolo12x.pt (see YOLOv12/args.yaml).
  • RT-DETRv2 and GDINO-SB. Both were fine-tuned and run with HuggingFace Transformers (Apache-2.0). Their config.json files do not record the base checkpoint name. The GDINO-SB architecture in config.json matches IDEA-Research/grounding-dino-base.
  • DINOv3. The paper uses DINOv3 only as a training-time teacher for RT-DETRv4. DINOv3 weights are under the DINOv3 License. Whether its terms extend to the distilled RT-DETRv4 checkpoint is unverified.
  • SAM 3. The SAM License covers the SAM 3 model weights and code, and works derived from them. Redistributed SAM Materials must be accompanied by a copy of the SAM License. The licence also lists prohibited uses.
  • Dataset. The StomataBench images and annotations are released under CC-BY-NC-4.0. That is the licence of the dataset, not of these weights.

Citation

@inproceedings{gummi2026stomatabench,
  title     = {StomataBench: Measuring Taxonomic Generalization in Stomatal Detection},
  author    = {Gummi, Sainath Reddy and Zhang, Hankui K. and Pack, Chulwoo and Solanki, Shyam and Chang, Young},
  booktitle = {Advances in Neural Information Processing Systems},
  volume    = {39},
  year      = {2026}
}

References

  1. Ren et al. (2015). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. Advances in Neural Information Processing Systems.
  2. Sun, Sun, Chen (2024). The evolution of object detection methods. Engineering Applications of Artificial Intelligence. https://doi.org/10.1016/j.engappai.2024.108458
  3. Wu et al. (2019). Detectron2. https://github.com/facebookresearch/detectron2
  4. Lin et al. (2017). Focal Loss for Dense Object Detection. IEEE International Conference on Computer Vision (ICCV). https://doi.org/10.1109/ICCV.2017.324
  5. Jocher, Chaurasia, Qiu (2023). Ultralytics YOLOv8. https://github.com/ultralytics/ultralytics
  6. Lv et al. (2024). RT-DETRv2: Improved Baseline with Bag-of-Freebies for Real-Time Detection Transformer. arXiv preprint arXiv:2407.17140.
  7. Liao et al. (2026). RT-DETRv4: Painlessly Furthering Real-Time Object Detection with Vision Foundation Models. European Conference on Computer Vision. https://doi.org/10.1007/978-3-032-37029-7_7
  8. Siméoni et al. (2025). DINOv3. arXiv preprint arXiv:2508.10104.
  9. Liu et al. (2024). Grounding DINO: Marrying DINO with Grounded Pre-training for Open-Set Object Detection. European Conference on Computer Vision. https://doi.org/10.1007/978-3-031-72970-6_3
  10. Carion et al. (2026). SAM 3: Segment Anything with Concepts. International Conference on Learning Representations.
  11. Tian, Ye, Doermann (2025). YOLOv12: Attention-Centric Real-Time Object Detectors. Advances in Neural Information Processing Systems. https://doi.org/10.52202/085713-2627
  12. Wolf et al. (2020). Transformers: State-of-the-Art Natural Language Processing. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. https://aclanthology.org/2020.emnlp-demos.6/
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Dataset used to train Sainath001/stomatabench_neurips_weights

Papers for Sainath001/stomatabench_neurips_weights