language:
- en
library_name: open_clip
pipeline_tag: zero-shot-image-classification
tags:
- biomedical
- vision-language
- contrastive-learning
- image-text-retrieval
- zero-shot-classification
license: other
BioCLIP
BioCLIP is a biomedical vision--language model with a ViT-L/14 image encoder and a BioClinical ModernBERT text encoder. It supports up to 512 text tokens.
Repository layout
BioClinicalModernBERT-large-ViT-L-14-pretrained.json
load_model.py
checkpoints/ five released training variants
evaluation/
|-- data/ BioMedFlickr and the unified 46-task validation data
|-- code/ two inference entry points and metric aggregation
`-- results/ released checkpoint results
Within the unified classification set, all benchmark files are under
evaluation/data/zero_shot_classification/datasets/: shared payloads live in
payloads/, while domain/dataset folders contain task definitions and any
dataset-specific files.
The model JSON is required by OpenCLIP to construct the architecture. The
model used by load_model.py and both evaluation runners is the recommended
checkpoint:
checkpoints/openclip_init_10epoch_best.pt
Evaluation
Both commands download the checkpoint and the required validation data from
this repository. A local data directory can also be supplied to the 46-task
runner with --data-root.
pip install -r evaluation/requirements.txt
python evaluation/code/evaluate_biomedflickr.py \
--output evaluation/results/biomedflickr_reproduced.json
python evaluation/code/evaluate_zero_shot_classification.py \
--output evaluation/results/unified46_reproduced.json
Results
Complete checkpoint results and public-model comparisons are reported in
evaluation/RESULTS.md. Machine-readable outputs are
stored in evaluation/results/.
Intended use
BioCLIP is intended for research and is not a clinical device.