v802-classifier / README.md
sneo's picture
Add v802 classifier head and source code
59839e3 verified
|
Raw History Blame Contribute Delete
2.38 kB
---
library_name: scikit-learn
pipeline_tag: image-classification
tags:
- multi-label
- dinov3
---
# v802 classifier
Multi-label image classifier that predicts which of six characters appear in an image:
Applejack, Fluttershy, Pinkie Pie, Rainbow Dash, Rarity, Twilight Sparkle.
- Backbone (frozen, **not included**): DINOv3 ViT-L/16, `facebook/dinov3-vitl16-pretrain-lvd1689m`
- Head (`assets/v802/classifier.joblib`): ensemble of a MIL head on patch tokens from layers 16/20/24 (weight 0.6)
and a per-class logistic-regression probe (weight 0.4), followed by a label-set decoder with a label-set prior and
a cardinality model. The output is a set of labels plus per-class probabilities.
- Input: RGB resized to 640x352, inferred on the original and its horizontal flip. Runs on CPU.
## Files
| Path | Contents |
|---|---|
| `assets/v802/classifier.joblib` | Trained head bundle (`classes`, `components`, `config`, `decoder`) |
| `src/` | Inference, training and experiment code |
| `requirements.txt` | Pinned dependency versions used in production |
`classifier.joblib` is a pickle. Loading it executes code, so only load it from a source you trust.
## Setup
1. Request access to `facebook/dinov3-vitl16-pretrain-lvd1689m` on Hugging Face (gated under the DINOv3 license),
then download it:
```bash
hf download facebook/dinov3-vitl16-pretrain-lvd1689m --local-dir models/dinov3-vitl16
```
2. Install dependencies (torch is the CPU build):
```bash
pip install --extra-index-url https://download.pytorch.org/whl/cpu -r requirements.txt
```
## Inference
```bash
python src/predict_v802.py \
--bundle assets/v802/classifier.joblib \
--model models/dinov3-vitl16 \
image1.jpg images_dir/
```
Options: `--threads N` (default 4), `--fp32` (disable bf16 autocast), `--output out.jsonl`.
Each line of output is JSON: `{"file": ..., "labels": [...], "probabilities": {class: p, ...}}`.
`src/serve_v802.py` takes the same `--bundle` / `--model` arguments and serves a minimal HTTP endpoint (`--port`, default 8802).
## Notes
- Training and experiment scripts (`train_*`, `experiment_*`, `extract_features_*`, `build_v802.py`) expect an
`artifacts/` tree of datasets, splits and features that is not included, so they will not run as-is.
- Training data: 736 labeled dev images plus two 32-image holdout sets (not included).