| --- |
| language: |
| - en |
| library_name: open_clip |
| pipeline_tag: zero-shot-image-classification |
| tags: |
| - biomedical |
| - vision-language |
| - contrastive-learning |
| - image-text-retrieval |
| - zero-shot-classification |
| license: other |
| --- |
| |
| # BioCLIP |
|
|
| BioCLIP is a biomedical vision--language model with a ViT-L/14 image encoder |
| and a BioClinical ModernBERT text encoder. It supports up to 512 text tokens. |
|
|
| ## Repository layout |
|
|
| ```text |
| BioClinicalModernBERT-large-ViT-L-14-pretrained.json |
| load_model.py |
| checkpoints/ five released training variants |
| evaluation/ |
| |-- data/ BioMedFlickr and the unified 46-task validation data |
| |-- code/ two inference entry points and metric aggregation |
| `-- results/ released checkpoint results |
| ``` |
|
|
| Within the unified classification set, all benchmark files are under |
| `evaluation/data/zero_shot_classification/datasets/`: shared payloads live in |
| `payloads/`, while domain/dataset folders contain task definitions and any |
| dataset-specific files. |
|
|
| The model JSON is required by OpenCLIP to construct the architecture. The |
| model used by `load_model.py` and both evaluation runners is the recommended |
| checkpoint: |
|
|
| [`checkpoints/openclip_init_10epoch_best.pt`](checkpoints/openclip_init_10epoch_best.pt) |
|
|
| ## Evaluation |
|
|
| Both commands download the checkpoint and the required validation data from |
| this repository. A local data directory can also be supplied to the 46-task |
| runner with `--data-root`. |
|
|
| ```bash |
| pip install -r evaluation/requirements.txt |
| |
| python evaluation/code/evaluate_biomedflickr.py \ |
| --output evaluation/results/biomedflickr_reproduced.json |
| |
| python evaluation/code/evaluate_zero_shot_classification.py \ |
| --output evaluation/results/unified46_reproduced.json |
| ``` |
|
|
| ## Results |
|
|
| Complete checkpoint results and public-model comparisons are reported in |
| [`evaluation/RESULTS.md`](evaluation/RESULTS.md). Machine-readable outputs are |
| stored in [`evaluation/results/`](evaluation/results/). |
|
|
| ## Intended use |
|
|
| BioCLIP is intended for research and is not a clinical device. |
|
|
|
|
