README.md

---
license: openmdw-1.0
library_name: transformers
pipeline_tag: feature-extraction
tags:
- vision-language
- multimodal
- image-text-retrieval
- hyperbolic-embeddings
---

<div align="center">
  <a href="https://hyper3labs.com">
    <img src="https://hyper3labs.com/brand-assets/hyper3labs-logo-primary.svg" width="64" height="64" alt="hyper³labs logo">
  </a>
  <p><strong>hyper³labs</strong></p>
  <h1>hyper3-clip-v1</h1>
  <p>
    <a href="https://arxiv.org/abs/2608.29313">Research paper</a> ·
    <a href="https://hyper3labs.com">Website</a> ·
    <a href="https://discord.gg/Qf2pXtY4Vf">Discord</a>
  </p>
</div>

**hyper3-clip-v1** is a hyperbolic vision–language model for visual search. It brings images and text into a shared embedding space for broad-to-specific retrieval and compositional matching.

- **Broad-to-specific retrieval:** Search for a broad concept such as “animal” to find different types of animals, then narrow your search with a specific description.
- **Compositional matching:** Find images by combinations of objects, attributes, and relationships—for example, “a red car beside a tree.” This helps distinguish similar scenes where a color, object, or relationship changes.

## Results

| Model | [SugarCrepe](https://github.com/RAIVNLab/sugar-crepe) macro accuracy |
|---|---:|
| **hyper3-clip-v1** | **79.54%** |
| Jina CLIP v2 | 75.02% |
| OpenAI CLIP ViT-B/16 | 73.06% |

Image retrieval — mAP (%) on [evaluation subsets](https://hyper3labs.com/) of each dataset.

| Dataset | hyper3-clip-v1 | OpenAI CLIP ViT-B/32 |
|---|---:|---:|
| Amazon Berkeley Objects — product types | **58.2** | 55.2 |
| DeepFashion In-Shop — same item | **63.5** | 35.2 |
| COCO — object categories | **55.4** | 53.2 |

## Quick start

Complete the short access form on this page, then install and sign in.

```bash
pip install "torch>=2.2" "transformers>=4.49,<5" "timm>=1.0" \
  "safetensors>=0.4" "Pillow>=10" "huggingface_hub>=0.34"
hf auth login
```

For SentenceTransformers, also run `pip install "sentence-transformers>=5.5.1,<6"`.

This example ranks three descriptions of a sofa. The example image downloads automatically.

```python
from PIL import Image
from huggingface_hub import hf_hub_download
from transformers import AutoModel

model_id = "hyper3labs/hyper3-clip-v1"
model = AutoModel.from_pretrained(model_id, trust_remote_code=True).eval()
image = Image.open(hf_hub_download(model_id, "examples/grey-velvet-sofa.jpg")).convert("RGB")

descriptions = ["a grey velvet sofa", "a blue velvet sofa", "a wooden chair"]
scores = model.score(image, descriptions)

for score, text in sorted(zip(scores.tolist(), descriptions), reverse=True):
    print(f"{score:.3f}  {text}")
```

The default is Lorentz scoring. Scores are negative; higher means a closer match.

To try your own, use `image = Image.open("your-image.jpg").convert("RGB")` and change the descriptions — broad words such as “furniture” work alongside specific ones.

## Scoring

Use `scoring="lorentz"`, `"cosine"`, or `"cone"` with `model.score(image, descriptions, scoring=...)`.

| Scoring | What it measures | Use |
|---|---|---|
| **Lorentz** (default) | Closeness in hyperbolic space; gives the same ranking as hyperbolic distance | General image–text retrieval |
| **Cosine** | Similarity of embedding directions | Standard vector similarity search |
| **Cone** | How well an image fits a description's general-to-specific region | Directional and compositional matching |

Higher is better for all three. Their numerical scales differ. The SugarCrepe result above uses cone scoring.

<details>
<summary><strong>Use with Qdrant</strong></summary>

Install the client and start a local Qdrant server:

```bash
pip install "qdrant-client>=1.19,<2"
docker run --rm -p 127.0.0.1:6333:6333 qdrant/qdrant:v1.19.1
```

In Python, continue from the quick start:

```python
from qdrant_client import QdrantClient

index = model.qdrant(QdrantClient(url="http://localhost:6333"), "hyper3_images")
index.create_collection()
index.upsert([image], ids=[1], payloads=[{"label": "grey velvet sofa"}])

for hit in index.search("a grey sofa", limit=5):
    print(hit.id, hit.score, hit.payload["label"])
```

The helper stores one search vector and two geometry values per image. Qdrant finds candidates with its index, then ranks them by Lorentz score in the same request. Only the final results are returned.

For a hosted deployment, pass your server URL and API key to `QdrantClient`. `index.search` accepts `query_filter` for filtering and `candidates` (default 200) to tune shortlist recall. For image-to-image search, pass an `encode_image_lorentz` embedding to `index.search_embedding`. Create payload indexes for your filters before bulk ingestion. Use a separate collection for each model revision.

</details>

## License and community

hyper3-clip-v1 is free to use under [OpenMDW-1.0](https://huggingface.co/hyper3labs/hyper3-clip-v1/blob/main/LICENSE). The short access form helps us understand how people use the model.

[Discord](https://discord.gg/Qf2pXtY4Vf) · [Website](https://hyper3labs.com) · [Attribution notice](https://huggingface.co/hyper3labs/hyper3-clip-v1/blob/main/NOTICE)