Zero-Shot Image Classification
Transformers
Safetensors
siglip
siglip2
image-text-retrieval
furniture
interior-design
home-decor
Instructions to use AlphJain/home-decor-siglip2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use AlphJain/home-decor-siglip2 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("zero-shot-image-classification", model="AlphJain/home-decor-siglip2") pipe( "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/hub/parrots.png", candidate_labels=["animals", "humans", "landscape"], )# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForZeroShotImageClassification processor = AutoProcessor.from_pretrained("AlphJain/home-decor-siglip2") model = AutoModelForZeroShotImageClassification.from_pretrained("AlphJain/home-decor-siglip2", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Home-decor SigLIP 2 (v3-clean-nostyle)
A fine-tune of google/siglip2-base-patch16-224 for
furniture and home-decor search: real room photo β product, text β product, and text β room.
Code (data cleaning, training, evaluation): github.com/jainalphin/home-decor-siglip. Every evaluation table, including runs that did not help: RESULTS.md.
Results
All numbers are on held-out data that was never used for training or for choosing the checkpoint.
| Task | Base SigLIP 2 | This model |
|---|---|---|
| Real room photo β product (Shop the Look, 2,250 queries, 19k-product gallery), R@1 / R@10 | 0.076 / 0.364 | 0.164 / 0.508 |
| Real furniture photo β other photos of the same model (Pix3D, 6,795 queries), R@1 | 0.778 | 0.827 |
| Search query β room (DeepFurniture test rooms with no near-copy in training), R@10 | 0.281 | 0.391 |
| Search query β furniture crop (DeepFurniture test), R@10 | 0.361 | 0.440 |
| Search query β product photo (ABO test), R@1 | 0.620 | 0.680 |
95% bootstrap intervals for the "this model β base" differences exclude zero for the first four rows. For ABO only R@10 was bootstrapped (+0.009, interval [0.000, 0.018]); both models are near the ceiling there.
Usage
import torch
from PIL import Image
from transformers import AutoModel, AutoProcessor
model_id = "AlphJain/home-decor-siglip2"
model = AutoModel.from_pretrained(model_id).eval()
processor = AutoProcessor.from_pretrained(model_id)
image = Image.open("room.jpg").convert("RGB")
texts = ["green velvet sofa", "scandinavian bedroom", "brass floor lamp"] # lowercase, as in training
with torch.no_grad():
img = model.get_image_features(**processor(images=image, return_tensors="pt"))
txt = model.get_text_features(**processor(text=texts, padding="max_length", max_length=64,
truncation=True, return_tensors="pt"))
img = torch.nn.functional.normalize(img, dim=-1)
txt = torch.nn.functional.normalize(txt, dim=-1)
scores = (img @ txt.T)[0]
for text, score in sorted(zip(texts, scores.tolist()), key=lambda x: -x[1]):
print(f"{score:.3f} {text}")
- Downloads last month
- 41
Model tree for AlphJain/home-decor-siglip2
Base model
google/siglip2-base-patch16-224