StomataBench: Measuring Taxonomic Generalization in Stomatal Detection
Sainath Reddy Gummi, Hankui K. Zhang, Chulwoo Pack, Shyam Solanki, Young Chang
This repository holds the 15 fine-tuned stomatal detectors of the paper. Each was fine-tuned once on OD-Train with a single stomata class.
| Paper | https://openreview.net/forum?id=kUdZ3znDlI |
| Project page | https://gummi001.github.io/StomataBench/ |
| DOI | https://doi.org/10.57967/hf/11473 |
| Dataset | https://huggingface.co/datasets/Sainath001/StomataBench (DOI https://doi.org/10.57967/hf/8667) |
| Code | https://github.com/gummi001/stomatabench_neurips_eval |
| Licence | Set per model, see Licences |
What this repository holds
One folder per detector. The folder names are the model IDs of the paper, with x in place of ×.
| Model ID | Folder | Weight file | Config file(s) | Framework | Licence |
|---|---|---|---|---|---|
| FRCNN-R50-1× | FRCNN-R50-1x/ |
StomataBench-OD_FASTER-R-CNN-R50-FPN-1x.pth |
config3.yaml |
Detectron2 | CC-BY-SA-3.0 (weights), Apache-2.0 (code) |
| FRCNN-R50-3× | FRCNN-R50-3x/ |
StomataBench-OD_FASTER-R-CNN-R50-FPN-3x.pth |
config3.yaml |
Detectron2 | CC-BY-SA-3.0 (weights), Apache-2.0 (code) |
| FRCNN-R101-1× | FRCNN-R101-1x/ |
StomataBench-OD_FASTER-R-CNN-R101-FPN-1x.pth |
config3.yaml |
Detectron2 | CC-BY-SA-3.0 (weights), Apache-2.0 (code) |
| FRCNN-X101 | FRCNN-X101/ |
StomataBench-OD_FRCNN-X101.pth |
config.yaml |
Detectron2 | CC-BY-SA-3.0 (weights), Apache-2.0 (code) |
| Retina-R50-1× | Retina-R50-1x/ |
StomataBench-OD_Retina-R50-1x.pth |
config5.yaml |
Detectron2 | CC-BY-SA-3.0 (weights), Apache-2.0 (code) |
| Retina-R50-3× | Retina-R50-3x/ |
StomataBench-OD_RETINA-NET-RESNET50-FPN-3x.pth |
config5.yaml |
Detectron2 | CC-BY-SA-3.0 (weights), Apache-2.0 (code) |
| Retina-R101-3× | Retina-R101-3x/ |
StomataBench-OD_RETINA-NET-RESNET101-FPN-3x.pth |
config5.yaml |
Detectron2 | CC-BY-SA-3.0 (weights), Apache-2.0 (code) |
| YOLOv8 | YOLOv8/ |
StomataBench-OD_YOLOv8.pt |
args.yaml |
Ultralytics | AGPL-3.0 |
| YOLOv11 | YOLOv11/ |
StomataBench-OD_YOLOv11.pt |
args.yaml |
Ultralytics | AGPL-3.0 |
| YOLOv12 | YOLOv12/ |
StomataBench-OD_YOLOv12.pt |
args.yaml |
Ultralytics | AGPL-3.0 |
| YOLOv26 | YOLOv26/ |
StomataBench-OD_YOLOv26.pt |
args.yaml |
Ultralytics | AGPL-3.0 |
| RT-DETRv2 | RT-DETRv2/ |
model.safetensors |
config.json, preprocessor_config.json |
HuggingFace Transformers | Apache-2.0 |
| RT-DETRv4 | RT-DETRv4/ |
StomataBench-OD_RT-DETRv4.pth |
none | Official RT-DETRv4 code | Apache-2.0 (DINOv3 teacher: see Licences) |
| GDINO-SB | GDINO-SB/ |
model.safetensors |
config.json, preprocessor_config.json |
HuggingFace Transformers | Apache-2.0 |
| SAM3-OD | SAM3-OD/ |
StomataBench-OD_SAM3.pt |
config.yaml, config_resolved.yaml |
Official SAM 3 code | SAM License |
The RT-DETRv2/ folder also holds the HuggingFace Trainer state files optimizer.pt, rng_state.pth, scaler.pt, scheduler.pt, trainer_state.json, training_args.bin. They are not needed for inference.
The RT-DETRv4/ folder holds the checkpoint only.
This card reports no results. In the paper, the detectors are evaluated unchanged on OD-Valid and on four held-out test sets (T-MAWP, T-DCWP, T-MSWP, T-GSH), with COCO localization metrics and with count and density measures. The results are in the paper and in the dataset repository.
Loading
Minimal examples. The scripts used in the paper are in the code repository.
from huggingface_hub import hf_hub_download
repo = "Sainath001/stomatabench_neurips_weights"
# Ultralytics (YOLOv8, YOLOv11, YOLOv12, YOLOv26)
from ultralytics import YOLO
yolo = YOLO(hf_hub_download(repo, "YOLOv8/StomataBench-OD_YOLOv8.pt"))
# Detectron2 (Faster R-CNN and RetinaNet folders)
from detectron2.config import get_cfg
from detectron2.engine import DefaultPredictor
cfg = get_cfg()
cfg.merge_from_file(hf_hub_download(repo, "FRCNN-R50-1x/config3.yaml"))
cfg.MODEL.WEIGHTS = hf_hub_download(repo, "FRCNN-R50-1x/StomataBench-OD_FASTER-R-CNN-R50-FPN-1x.pth")
predictor = DefaultPredictor(cfg)
# HuggingFace Transformers (RT-DETRv2)
from transformers import AutoImageProcessor, AutoModelForObjectDetection
processor = AutoImageProcessor.from_pretrained(repo, subfolder="RT-DETRv2")
rtdetr = AutoModelForObjectDetection.from_pretrained(repo, subfolder="RT-DETRv2")
GDINO-SB loads with AutoModelForZeroShotObjectDetection.from_pretrained(repo, subfolder="GDINO-SB"). Its folder has no tokenizer files, so the processor comes from IDEA-Research/grounding-dino-base (AutoProcessor.from_pretrained("IDEA-Research/grounding-dino-base")), as recorded in the run record of every GDINO-SB prediction file (processor_id). RT-DETRv4 and SAM3-OD need the official RT-DETRv4 and SAM 3 code. The inference scripts and the settings used in the paper are in the code repository (inference/ and scripts/run_inference_*.sh).
Model families
The detectors span five architecture families. The descriptions below follow the paper's Related Work section.
- Two-stage convolutional detectors (Faster R-CNN [1, 2, 3]) generate class-agnostic region proposals with a Region Proposal Network and then refine these candidates using a second-stage classifier and box regressor. This proposal-refinement design remains a strong baseline for small-object detection because candidate regions are explicitly revisited before final prediction.
- One-stage convolutional detectors (RetinaNet [2, 3, 4]) remove the proposal stage and directly predict boxes and class probabilities over multi-scale feature maps, using focal loss to address foreground–background imbalance.
- YOLO-family detectors [5] represent a deployment-oriented branch of one-stage detection, emphasizing fast single-pass inference, efficient feature aggregation, and strong training-time augmentation. YOLO26-X further reflects the recent shift toward end-to-end, NMS-free real-time detection.
- Real-time deformable transformer detectors cast detection as set prediction. RT-DETRv2 and RT-DETRv4 [6, 7] extend this paradigm toward real-time inference through efficient multi-scale encoding and a deformable-attention decoder. RT-DETRv4 further incorporates training-time vision-foundation-model distillation, using DINOv3 as a teacher through Deep Semantic Injection and Gradient-guided Adaptive Modulation, while preserving inference-time efficiency [7, 8].
- Foundation-model-derived detectors recast category-specific detection as language-conditioned or concept-prompt-conditioned set prediction. Grounding DINO with a Swin-Base backbone (GDINO-SB) [9] represents language-aligned open-set detection. It combines a visual backbone with a BERT-based text encoder, fuses image and text features through cross-modal attention, and uses language-guided query selection followed by a cross-modality decoder to produce bounding boxes grounded to the input phrase. SAM 3 [10] is a promptable-concept vision foundation model. Unlike earlier SAM models, it includes an image-level detector that can localize all instances matching a concept prompt. It combines a vision encoder, text encoder, geometry encoder, DETR-style encoder–decoder, learned object queries, and a mask decoder. SAM3-OD denotes the paper's detection-only use of the official SAM 3 model rather than a modified architecture.
Including GDINO-SB and SAM3-OD lets StomataBench compare foundation-model-derived detectors with closed-set detectors under the same shifts, without isolating which of their components explains a difference.
Training
Fifteen open-source object detectors were evaluated, spanning five architecture families: two-stage convolutional detectors, one-stage convolutional detectors, YOLO-family detectors, real-time deformable transformer detectors, and foundation-model-derived detectors. The benchmark includes Faster R-CNN variants with ResNet and ResNeXt backbones, RetinaNet, YOLOv8/11/12/26, RT-DETRv2, RT-DETRv4, GDINO-SB, and SAM3-OD [1, 2, 3, 4, 5, 6, 7, 9, 10, 11]. Model sizes range from 31.4M to 839.4M parameters. All detectors were fine-tuned once on OD-Train using a single stomata class and evaluated unchanged on every held-out test set. No model was given taxonomic labels, dataset identity, or test-set-specific tuning. Architectures, parameter counts, learning rates, training schedules, and batch sizes are summarized in the training-settings table below (Table S7 of the paper).
Pre-processing and training. Each detector used its native framework pre-processing pipeline to avoid introducing model-specific custom data handling. Detectron2 models were trained from COCO-format annotations and used the standard shortest-side resize of 800 px with maximum size 1333 px and ImageNet BGR normalization [3]. Ultralytics YOLO models used YOLO-format annotations, letterboxed inputs to 1024×1024, and retained the default Ultralytics training augmentations [5]. RT-DETRv2 used the HuggingFace auto-processor with images passed at native 1280×1024 resolution and ImageNet RGB normalization [6]. RT-DETRv4 used its official training configuration with 640×640 RGB inputs [7, 8]. GDINO-SB used the official HuggingFace processor with the text prompt "a stomata ." [9, 12]. SAM3-OD used the ODinW text-only configuration with 1008×1008 inputs and the prompt "stomata" during fine-tuning and inference [10]. All models were optimized with AdamW. All pretrained checkpoints were fine-tuned end-to-end on OD-Train.
Training settings of the detectors, all fine-tuned on OD-Train with a single stomata class. Params: total parameters in millions. LR: peak learning rate. Schedule: total training budget. BS: batch size per step. Every GPU has 80 GB.
| Model ID | Architecture | Params (M) | LR | Schedule | BS | Train GPU |
|---|---|---|---|---|---|---|
| Two-stage convolutional detectors [1, 3] | ||||||
| FRCNN-R50-1× | Faster R-CNN, ResNet-50-FPN, 1× | 41.4 | 5×10-5 | 25,000 iters | 4 | 1×A100 |
| FRCNN-R50-3× | Faster R-CNN, ResNet-50-FPN, 3× | 41.4 | 5×10-5 | 25,000 iters | 4 | 1×A100 |
| FRCNN-R101-1× | Faster R-CNN, ResNet-101-FPN, 1× | 60.4 | 5×10-5 | 25,000 iters | 4 | 1×A100 |
| FRCNN-X101 | Faster R-CNN, ResNeXt-101-FPN, 3× | 104.8 | 5×10-5 | 25,000 iters | 4 | 1×A100 |
| One-stage convolutional detectors [3, 4] | ||||||
| Retina-R50-1× | RetinaNet, ResNet-50-FPN, 1× | 38.0 | 5×10-5 | 25,000 iters | 4 | 1×A100 |
| Retina-R50-3× | RetinaNet, ResNet-50-FPN, 3× | 38.0 | 5×10-5 | 25,000 iters | 4 | 1×A100 |
| Retina-R101-3× | RetinaNet, ResNet-101-FPN, 3× | 57.1 | 5×10-5 | 25,000 iters | 4 | 1×A100 |
| YOLO-family detectors [5, 11] | ||||||
| YOLOv8 | YOLOv8-X | 68.2 | 8.7×10-6 | 400 epochs | 14 | 2×A100 |
| YOLOv11 | YOLOv11-X | 56.9 | 8.7×10-6 | 400 epochs | 14 | 2×A100 |
| YOLOv12 | YOLOv12-X | 59.1 | 8.7×10-6 | 400 epochs | 14 | 2×A100 |
| YOLOv26 | YOLOv26-X | 58.8 | 8.7×10-6 | 400 epochs | 14 | 2×A100 |
| Transformer detectors: RT-DETRv2 [6], RT-DETRv4 [7, 8] | ||||||
| RT-DETRv2 | RT-DETRv2, ResNet-50 | 42.9 | 5×10-5 | 50 epochs | 8 | 1×A100 |
| RT-DETRv4 | RT-DETRv4-L, HGNetV2-B4, DINOv3 teacher | 31.4 | 2.5×10-4 | 30 epochs | 16 | 1×H100 |
| Foundation-model-derived detectors: GDINO-SB [9], SAM3-OD [10] | ||||||
| GDINO-SB | Grounding-DINO, Swin-Base | 232.8 | 2×10-5 | 50 epochs | 4 | 1×A100 |
| SAM3-OD | SAM3, ODinW text-only configuration | 839.4 | 8×10-6 | 60 epochs | 32 | 2×H100 |
Training data: OD-Train
All 15 detectors were fine-tuned on OD-Train only.
The training set consists of 3,154 microscopy images and 134,802 annotated stomatal complexes across 21 species in 10 botanical families (Table S1 and Figure S1 of the paper, reproduced below). Coverage is concentrated in Rosales, with secondary representation from Ericales, Cornales, Ranunculales, Vitales, Fagales, Sapindales, Dipsacales, and Laurales. Each stomatal complex is annotated as a single-class (stomata) axis-aligned bounding box.
Imaging and annotation protocol. OD-Train images were sampled from living accessions at the Arnold Arboretum of Harvard University, Boston. All images were captured by the authors using a ProScope HR handheld digital microscope (USB 2.0, 400× magnification) connected to a tripod-mounted laptop running ProScope HR v1.2.1. Image resolution is 1280×1024 px, with calibration scale s = 0.5988 µm px-1 (167 px per 0.1 mm) measured using a stage micrometer. Imaging was performed on the abaxial (lower) leaf surface. The clear contact dome was gently pressed against the leaf to stabilize the field of view, and the integrated LED ring provided coaxial fill illumination. Field sessions were conducted under natural daylight, and images with strong specular highlights from direct solar reflections were excluded during annotation. For hardwood species, the lowest accessible branch was sampled. Visible stomata were annotated as bounding boxes in RectLabel Pro under the supervision of a botanist. No image augmentation was applied before annotation.
OD-Train species composition (21 species across 10 families). Mean density per species ranges from 30.92 to 277.27 stomata mm-2.
| Accession ID(s) | Scientific name | Family | Order | Images (N) | Stomata (N) | GT density (mm-2) |
|---|---|---|---|---|---|---|
| 403-97*C & 905-85*A | Actinidia arguta | Actinidiaceae | Ericales | 161 | 4,662 | 61.61 |
| 257-2006*B | Actinidia macrosperma | Actinidiaceae | Ericales | 145 | 5,110 | 74.99 |
| 107-2004*B | Calycanthus 'Venus' | Calycanthaceae | Laurales | 55 | 3,165 | 122.44 |
| 1690-77*B | Clematis terniflora | Ranunculaceae | Ranunculales | 308 | 6,900 | 47.67 |
| 276*A | Cotinus coggygria | Anacardiaceae | Sapindales | 56 | 5,285 | 200.81 |
| 14714-1*A | Hydrangea paniculata 'Praecox' | Hydrangeaceae | Cornales | 434 | 6,306 | 30.92 |
| 427-2016*A | Malus 'Blanche Ames' | Rosaceae | Rosales | 134 | 10,609 | 168.46 |
| 1951-80*A, 1951-80*B, 210-2001*A, 608-2016*A & 12198*B | Malus hupehensis | Rosaceae | Rosales | 240 | 12,063 | 106.95 |
| 28-2013*A | Malus ioensis | Rosaceae | Rosales | 49 | 4,966 | 215.64 |
| 79-2009*B, 51-2009*A, 78-2009*B, 48-2009*A, 49-2009*A, 79-2009*A, 75-2009*A & 74-2009*B | Malus sieversii | Rosaceae | Rosales | 204 | 12,591 | 131.33 |
| 49-98*A, 46-98*A, 228-2013*B & 228-2013*C | Malus spontanea | Rosaceae | Rosales | 128 | 6,724 | 111.77 |
| 80-2017C, 47-98*B & 50-2011B | Malus toringoides | Rosaceae | Rosales | 62 | 4,685 | 160.78 |
| 510-94*A | Malus × zumi | Rosaceae | Rosales | 172 | 7,846 | 97.06 |
| 596-86*A | Prinsepia sinensis | Rosaceae | Rosales | 226 | 6,552 | 61.69 |
| 119-96*A | Prunus 'Washi-no-o' | Rosaceae | Rosales | 75 | 3,712 | 105.31 |
| 13-2016*A | Quercus rubra | Fagaceae | Fagales | 116 | 15,116 | 277.27 |
| 80-2012*A | Rosa 'Queen Elizabeth' | Rosaceae | Rosales | 63 | 1,533 | 51.78 |
| 713-2010*MASS | Rosa 'Seafoam' | Rosaceae | Rosales | 162 | 7,513 | 98.68 |
| 531-34*C | Stewartia gemmata | Theaceae | Ericales | 54 | 1,308 | 51.54 |
| 158-2004*A | Viburnum sargentii 'Onondaga' | Adoxaceae | Dipsacales | 55 | 1,029 | 39.81 |
| 332-84*A | Vitis vinifera × amurensis 'Beihong' | Vitaceae | Vitales | 255 | 7,127 | 59.47 |
| Total (21 species, 10 families) | 3,154 | 134,802 |
Representative stomata micrographs of diverse woody plants from OD-Train (300 µm scale bar).
Intended use and limitations
- The checkpoints are released for research use, to reproduce the paper and to compare stomatal detectors under distribution shift.
- Benchmark performance is not a deployment guarantee. Target-domain calibration, confidence selection on independent validation data, and human review remain necessary.
- The detectors predict one class,
stomata, as axis-aligned bounding boxes of stomatal complexes. They do not distinguish guard cells from pavement cells, annotate pores, estimate aperture state, or measure pore-level geometry. - OD-Train is taxonomically broad within woody eudicots (plus one magnoliid) but does not include monocots, gymnosperms, ferns, or basal angiosperms (the ANA grade).
- OD-Train images come from one microscope and one annotation protocol. They were collected without controlled CO₂, humidity, water status, light history, developmental stage, or time-of-day metadata.
Licences
Each fine-tuned checkpoint inherits the licence of the upstream code and pretrained weights it was built from. The licences below were checked against the official sources listed. Redistribution must follow the terms of each upstream licence.
| Checkpoints | Licence of the fine-tuned weights | Official sources |
|---|---|---|
| FRCNN-R50-1x, FRCNN-R50-3x, FRCNN-R101-1x, FRCNN-X101, Retina-R50-1x, Retina-R50-3x, Retina-R101-3x | CC-BY-SA-3.0 (pretrained weights from the Detectron2 Model Zoo). Detectron2 code is Apache-2.0. | Detectron2 LICENSE, Detectron2 Model Zoo, License section |
| YOLOv8, YOLOv11, YOLOv26 | AGPL-3.0 (Ultralytics Enterprise License as the alternative) | Ultralytics LICENSE, Ultralytics licensing, YOLO26 docs |
| YOLOv12 | AGPL-3.0 | Official YOLOv12 repository, Ultralytics LICENSE |
| RT-DETRv2 | Apache-2.0 | Official RT-DETR / RT-DETRv2 repository, HuggingFace Transformers LICENSE, PekingU/rtdetr_v2_r50vd |
| RT-DETRv4 | Apache-2.0. DINOv3 teacher: DINOv3 License, see the note below. | Official RT-DETRv4 repository, DINOv3 License |
| GDINO-SB | Apache-2.0 | IDEA-Research/grounding-dino-base, Official Grounding DINO repository |
| SAM3-OD | SAM License | SAM 3 LICENSE, facebook/sam3 |
Notes on the table:
- Detectron2. Each Detectron2 config in this repository records
MODEL.WEIGHTSas an ImageNet-pretrained backbone from the Detectron2 Model Zoo, for exampledetectron2://ImageNetPretrained/MSRA/R-50.pkl. The Model Zoo states that all models available for download through it are licensed under CC-BY-SA-3.0. - Ultralytics. Ultralytics states that models trained with its code fall under AGPL-3.0 by default. YOLOv12 was trained with Ultralytics from
yolo12x.pt(seeYOLOv12/args.yaml). - RT-DETRv2 and GDINO-SB. Both were fine-tuned and run with HuggingFace Transformers (Apache-2.0). Their
config.jsonfiles do not record the base checkpoint name. The GDINO-SB architecture inconfig.jsonmatchesIDEA-Research/grounding-dino-base. - DINOv3. The paper uses DINOv3 only as a training-time teacher for RT-DETRv4. DINOv3 weights are under the DINOv3 License. Whether its terms extend to the distilled RT-DETRv4 checkpoint is unverified.
- SAM 3. The SAM License covers the SAM 3 model weights and code, and works derived from them. Redistributed SAM Materials must be accompanied by a copy of the SAM License. The licence also lists prohibited uses.
- Dataset. The StomataBench images and annotations are released under CC-BY-NC-4.0. That is the licence of the dataset, not of these weights.
Citation
@inproceedings{gummi2026stomatabench,
title = {StomataBench: Measuring Taxonomic Generalization in Stomatal Detection},
author = {Gummi, Sainath Reddy and Zhang, Hankui K. and Pack, Chulwoo and Solanki, Shyam and Chang, Young},
booktitle = {Advances in Neural Information Processing Systems},
volume = {39},
year = {2026}
}
References
- Ren et al. (2015). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. Advances in Neural Information Processing Systems.
- Sun, Sun, Chen (2024). The evolution of object detection methods. Engineering Applications of Artificial Intelligence. https://doi.org/10.1016/j.engappai.2024.108458
- Wu et al. (2019). Detectron2. https://github.com/facebookresearch/detectron2
- Lin et al. (2017). Focal Loss for Dense Object Detection. IEEE International Conference on Computer Vision (ICCV). https://doi.org/10.1109/ICCV.2017.324
- Jocher, Chaurasia, Qiu (2023). Ultralytics YOLOv8. https://github.com/ultralytics/ultralytics
- Lv et al. (2024). RT-DETRv2: Improved Baseline with Bag-of-Freebies for Real-Time Detection Transformer. arXiv preprint arXiv:2407.17140.
- Liao et al. (2026). RT-DETRv4: Painlessly Furthering Real-Time Object Detection with Vision Foundation Models. European Conference on Computer Vision. https://doi.org/10.1007/978-3-032-37029-7_7
- Siméoni et al. (2025). DINOv3. arXiv preprint arXiv:2508.10104.
- Liu et al. (2024). Grounding DINO: Marrying DINO with Grounded Pre-training for Open-Set Object Detection. European Conference on Computer Vision. https://doi.org/10.1007/978-3-031-72970-6_3
- Carion et al. (2026). SAM 3: Segment Anything with Concepts. International Conference on Learning Representations.
- Tian, Ye, Doermann (2025). YOLOv12: Attention-Centric Real-Time Object Detectors. Advances in Neural Information Processing Systems. https://doi.org/10.52202/085713-2627
- Wolf et al. (2020). Transformers: State-of-the-Art Natural Language Processing. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. https://aclanthology.org/2020.emnlp-demos.6/
