File size: 3,453 Bytes
326e3fb 70c5810 326e3fb 839fc2c 326e3fb 839fc2c 326e3fb 839fc2c 326e3fb | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 | ---
license: apache-2.0
tags:
- fashion
- retrieval
- text-to-image
- open_clip
- routing
pipeline_tag: feature-extraction
---
# MODA Duo
**Two open constituents, one answer per query.** Duo routes each text query to whichever
open MODA system suits its shape — short catalogue titles to
[MODA Pro Lite+](https://huggingface.co/HopitAI/moda-pro-lite-plus), longer descriptions to
[MODA](https://huggingface.co/HopitAI/moda-fashionsiglip-multiview-203m) — and runs
**one encoder and one nearest-neighbour query per search**.
Duo adds **zero parameters**. It is a serving recipe over two open systems, not a new model.
## Why
Fashion search queries come in two shapes, and no single small model is best at both:
| query shape | example | best open system ≤250M |
|---|---|---|
| catalogue title | `buckle round toe flat shoes` | MODA Pro Lite+ |
| natural description | `When warm weekends are abound, make sure your closet…` | MODA |
Duo picks per query. On a mixed workload it beats **both** constituents.
## Results
MAP@10, full corpus, all ground-truth queries, one evaluator (`pytrec_eval map_cut.10`),
paired bootstrap 10,000 resamples.
| benchmark | MODA | MODA Pro Lite+ | **MODA Duo** |
|---|---:|---:|---:|
| KAGL | 0.2887 | 0.3201 | **0.3201** |
| Polyvore | 0.3726 | 0.4049 | **0.4049** |
| Atlas | 0.1862 | 0.1904 | **0.1904** |
| Fashion200K | **0.1946** | 0.1846 | 0.1866 |
| DeepFashion In-Shop | **0.1642** | 0.1026 | 0.1640 |
| DeepFashion Multimodal | 0.0147 | 0.0133 | **0.0159** |
| **pooled, 12,000 queries** | 0.2035 | 0.2026 | **0.2137** |
Pooled across all six benchmarks — the mixed workload a router exists for — Duo is
**+5.0% over MODA and +5.4% over MODA Pro Lite+**, both significant.
Fashion200K is the honest miss: its queries sit where the two constituents are hardest to
tell apart, and Duo trails MODA there by 4%. Where a workload is known to be all long descriptions, use MODA
directly.
## Serving cost
```
indexes 2 one per constituent, built offline
stored vectors per item 2
encoders run per query 1 only the routed constituent's text tower
ANN queries per search 1
re-ranking none
```
Compared with a single open model, Duo costs one extra index at build time and nothing
extra at query time.
## Use
```bash
pip install open_clip_torch pillow numpy hnswlib
python serving_ann.py --demo
```
```python
from serving_ann import Duo
duo = Duo() # loads both constituents
duo.build(images) # encodes the catalogue with both, builds two indexes
ids, scores, routes = duo.search(["black leather ankle boots"], k=10)
```
The router is a callable — replace it with any policy that maps a query to a constituent:
```python
duo = Duo(router=lambda q: "moda" if looks_like_a_description(q) else "moda_pro_lite_plus")
```
## Evaluation
All figures are full corpus, all ground-truth queries, MAP@10 under one evaluator
(`pytrec_eval map_cut.10`), paired bootstrap with 10,000 resamples. Per-query results are in
the [repository](https://github.com/hopit-ai/Moda).
## Related
- [MODA](https://huggingface.co/HopitAI/moda-fashionsiglip-multiview-203m) — FashionSigLIP with a serving harness. Open source, open weights.
- [MODA Pro Lite](https://huggingface.co/HopitAI/moda-pro-lite) — a trained fashion encoder. Open weights.
- MODA Pro — hosted. Fuses both constituents rather than choosing between them.
|