File size: 4,608 Bytes
3eaff2e | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 | ---
license: apache-2.0
tags:
- fashion
- retrieval
- text-to-image
- open_clip
- siglip2
pipeline_tag: feature-extraction
library_name: open_clip
base_model: HopitAI/moda-pro-lite
---
# MODA Pro Lite+
**The strongest open system at ≤250M parameters on catalogue and title search.**
MODA Pro Lite+ is [MODA Pro Lite](https://huggingface.co/HopitAI/moda-pro-lite) — a 213M
fashion retrieval encoder — served with a calibrated multi-view recipe. This repository holds
the recipe, as runnable code; the weights are pulled from `moda-pro-lite` at load time.
Zero added parameters. One stored vector per item. The uplift is paid once at index time and
costs nothing at query time.
## Results
MAP@10, full corpus, all ground-truth queries, one evaluator (`pytrec_eval map_cut.10`).
`MODA` is FashionSigLIP with its own serving recipe, shown for reference.
| benchmark | MODA | Pro Lite (bare) | **Pro Lite+** (with recipe) |
|---|---:|---:|---:|
| KAGL | 0.2887 | 0.3055 | **0.3201** |
| Polyvore | 0.3726 | 0.3952 | **0.4049** |
| Atlas | 0.1862 | 0.1814 | **0.1904** |
| Fashion200K | **0.1946** | 0.1758 | 0.1846 |
| DeepFashion In-Shop | **0.1642** | 0.0930 | 0.1026 |
| DeepFashion Multimodal | **0.0147** | 0.0118 | 0.0133 |
**Pro Lite+ leads the ≤250M class on KAGL, Polyvore and Atlas** — +10.9% over MODA on KAGL,
+8.7% on Polyvore, both significant under a paired bootstrap (10,000 resamples).
The recipe is worth +2.5% to +12.8% over the bare encoder on every benchmark, and costs
nothing at query time: the views are fused into a single vector before indexing.
**Where this model is weak, stated plainly.** Pro Lite is tuned for short catalogue titles.
On long natural-language descriptions it trails FashionSigLIP substantially — DeepFashion
In-Shop queries average 75 words, and Pro Lite+ scores 0.1026 there against MODA's 0.1642.
If your queries are descriptions rather than titles, use
[MODA Duo](https://huggingface.co/HopitAI/moda-duo), which routes per query.
## Serving cost
```
stored vectors per item : 1
ANN queries per search : 1
image forwards at index : 3x offline, paid once
text forwards per query : 2x negligible beside the ANN probe
```
The recipe is a rule for *what you encode*, not a model change. Views are combined into one
unit vector before indexing, so nearest-neighbour search costs exactly what the bare encoder
costs — same index, same probe, no extra routes and no re-ranking.
## Use
```bash
pip install open_clip_torch pillow numpy hnswlib
python serving_ann.py --demo
```
```python
from serving_ann import load, encode_images, encode_queries, build_index, search
enc = load() # open_clip, this repo's weights
docs = encode_images(catalogue, enc) # (n, 768) float32, one vector per item
index = build_index(docs) # hnswlib, cosine via inner product
qry = encode_queries(["black leather ankle boots"], enc)
ids, scores = search(index, qry, k=10)
```
Bare encoder, if you would rather not use the recipe:
```python
import open_clip, torch
model, _, preprocess = open_clip.create_model_and_transforms("hf-hub:HopitAI/moda-pro-lite")
tokenizer = open_clip.get_tokenizer("hf-hub:HopitAI/moda-pro-lite")
model.eval()
with torch.no_grad():
image = torch.nn.functional.normalize(model.encode_image(preprocess(img).unsqueeze(0)), dim=-1)
text = torch.nn.functional.normalize(model.encode_text(tokenizer(["black leather ankle boots"])), dim=-1)
score = (text @ image.T).item()
```
768-d embeddings, cosine similarity, one vector per item. Index them in any vector database.
## The recipe
```
document = normalize(official + 0.25 * square_pad + 0.25 * foreground_pad)
query = normalize(raw + 0.25 * "a photo of {query}")
```
`serving_ann.py` implements it. Zero added parameters, one stored vector.
## Evaluation
All figures are full corpus, all ground-truth queries, MAP@10 under one evaluator
(`pytrec_eval map_cut.10`), float32. Per-query results and confidence intervals are in the
[repository](https://github.com/hopit-ai/Moda).
## Related
- [MODA Pro Lite](https://huggingface.co/HopitAI/moda-pro-lite) — the bare encoder these weights come from.
- [MODA Duo](https://huggingface.co/HopitAI/moda-duo) — routes each query to Pro Lite+ or MODA by its shape; beats both on a mixed workload.
- [MODA](https://huggingface.co/HopitAI/moda-fashionsiglip-multiview-203m) — FashionSigLIP with a serving recipe. Stronger on long descriptions.
- [MODA-SigLIP-Distilled](https://huggingface.co/HopitAI/moda-fashion-distilled) — image-to-image retrieval.
|