README.md

---
title: CLIP
emoji: ⚡
colorFrom: yellow
colorTo: pink
sdk: gradio
sdk_version: 6.30.0
python_version: '3.12'
app_file: app.py
pinned: false
---

# GeoCLIP + StreetCLIP — regional hypothesis scoring

Two **real, separately trained models** for the early stages of photo geolocation.
They do not generate overhead photographs, and their scores do not establish precise coordinates.

## What this Space runs

- **StreetCLIP:** compare an original ground-level photo with up to 100 English candidate descriptions. Returns ranked cosine similarities and logits. Its author documents that its training data excludes China and India; Chinese-town results require independent evaluation.
- **GeoCLIP:** score up to 100 caller-supplied GPS coordinates, or retrieve from the author's 100,000-coordinate gallery. Returned locations are *gallery hypotheses*, not measured camera positions.

CPU inference remains the default for all three existing inference APIs. This selected ZeroGPU runtime requires a registered GPU task at startup, so `/streetclip_score_gpu` provides a genuine optional CUDA implementation of StreetCLIP scoring (not a dummy/health task). It uses an independently staged copy of the same published weights and ordinary ZeroGPU quota, with a 30-second maximum function duration. No paid hardware changes, subscription changes, quota evasion, or automatic GPU retries are used. GPU quota/capacity errors are returned explicitly; the user may choose the existing CPU endpoint. Both CPU and GPU service remain subject to host resources, sleeping, scheduling, and availability.

### Download and storage boundary

The public repository contains only code and small metadata. On startup in `SammySnake/CLIP`, the HF container obtains approximately **3.5 GB** of pinned model weights from the original public model repositories. Models are not downloaded to the developer's computer. The application refuses model acquisition outside the exact target Space. Uploads are temporary, with Gradio cleanup configured for files older than one hour, checked every ten minutes; they are not saved as examples or committed. Do not upload private/sensitive material to a public demo.

## Named APIs (Gradio client)

```python
import json
from gradio_client import Client, handle_file

client = Client("SammySnake/CLIP")
print(client.predict(api_name="/status"))
# Optional: same arguments as /streetclip_score, api_name="/streetclip_score_gpu".
# That separate route uses normal ZeroGPU quota; the default APIs remain CPU.
# Wait for all_models_ready == True; loaded UI alone is not model readiness.

print(client.predict(
    handle_file("your-authorized-ground-photo.jpg"),
    json.dumps(["A street in Jiangsu, China.", "A street in Anhui, China."]),
    2, api_name="/streetclip_score",
))
print(client.predict(
    handle_file("your-authorized-ground-photo.jpg"), 5,
    api_name="/geoclip_predict",
))
print(client.predict(
    handle_file("your-authorized-ground-photo.jpg"),
    json.dumps([
        {"id": "region-a", "lat": 31.0, "lon": 120.0},
        {"id": "region-b", "lat": 33.0, "lon": 118.0},
    ]), 2, api_name="/geoclip_candidates",
))
```

`/status` reports actual model load phases, software/model revisions, CPU limits, cgroup memory limits, and optional GPU staging. GPU `ready` means the separate weights were staged on CUDA during startup, not that a hosted GPU inference has already passed validation. CPU models and the optional GPU copy are loaded synchronously before launching Gradio, as required by ZeroGPU startup; first startup can take several minutes. A started server is not equivalent to ready model weights. Load failures are reported as errors, never replaced by generic pretrained CLIP or fake scores.

### Contracts

- RGB input: 16×16 minimum, 16 megapixels maximum, upload limit 10 MB. EXIF orientation is applied; no generative enhancement.
- `top_k`: integer 1–20. Candidate arrays: 1–100 items, unique string IDs.
- StreetCLIP descriptions: English recommended, maximum 500 characters and the model's 77-token limit. Overlong captions are rejected, not silently truncated. Accepts JSON `list[str]` or `[{"id":"a","text":"..."}]`.
- GeoCLIP coordinates: GPS latitude then longitude, in degrees; latitude ±90, longitude ±180. No GCJ-02 or BD-09 conversion is performed.
- Queue length 8, inference concurrency 1. Errors explicitly report invalid input or model-not-ready state.
- `cosine_similarity` and `logit` are ranking scores. `gallery_relative_softmax` is relative to the author's fixed gallery, **not calibrated confidence of correctness**.

## How it fits the geolocation pipeline

1. GeoCLIP / StreetCLIP provide regional priors.
2. A separate cross-view model (e.g. Sample4Geo) ranks ground-photo / satellite candidates.
3. OrienterNet projects learned features into a BEV representation and matches semantic map features.
4. Independent real imagery and geometry verify a specific location.

Do **not** feed a color visualization of BEV feature channels as if it were a satellite photograph. Do not average uncalibrated raw scores from different models. A precise-looking coordinate is not proof of a precise location.

## Provenance and license

See [ATTRIBUTION.md](./ATTRIBUTION.md) and [model-sources.json](./model-sources.json).
StreetCLIP is **CC-BY-NC-4.0 (non-commercial)**; GeoCLIP is MIT. This is a research/evaluation demo, not a commercial deployment grant.

No location accuracy or runtime performance claim is established by building the app; hosted inference must be tested separately on authorized images with independent ground truth.