---
title: Web Graph kNN Spot-check
emoji: 🔎
colorFrom: indigo
colorTo: gray
sdk: gradio
sdk_version: 6.26.0
python_version: 3.12.12
app_file: app.py
pinned: false
short_description: Nearest neighbours of a host in the CC host-graph embedding
---
# Web graph kNN spot-check

Type a hostname, get its nearest neighbours in the learned 128-d Common Crawl host-graph
embedding. Companion to the 2D viewer: that one shows *where* hosts land after a UMAP
projection, this one answers *who is actually nearest* in the real space.

## Running locally

```bash
uv sync --extra faiss --extra viewer
wgl knn build --hf-repo commoncrawl/web-graph-embeddings --shards 8 --row-groups 1 \
    --out data/knn/cc_deg8_poc
WGL_KNN_INDEX=data/knn/cc_deg8_poc uv run python knn-viewer/app.py
```

## Running as a Space

The app reads its index from `$WGL_KNN_INDEX`, defaulting to `/data` — the mount path for a
HuggingFace volume. Point a **private** bucket at it:

```bash
hf buckets create commoncrawl/web-graph-knn-index --private
hf sync data/knn/cc_deg8_full hf://buckets/commoncrawl/web-graph-knn-index/cc_deg8 --delete
hf spaces volumes set commoncrawl/web-graph-knn \
    -v hf://buckets/commoncrawl/web-graph-knn-index:/data
```

`app.py` needs `wgl.knn.index`, which pulls only numpy, faiss and tldextract, so `wgl` is
**vendored** next to `app.py` on the Space rather than installed. Only the *import closure* of
`app.py` ships — 13 modules of 100, so the Space holds the code it runs and not the training
tree. `KnnIndex` stays a `wgl` module rather than being copied into the app on purpose: it reads
the artifact format `wgl.knn.build` writes, `wgl knn query` reads the same one, and
`tests/test_knn_index.py` round-trips build against load to keep them honest. Deploying does the
vendoring for you:

```bash
make knn-diff     # which files the Space is missing or has stale (changes nothing)
make knn-deploy   # upload the app + vendored src/wgl, skipping what the Space already has
make knn-serve    # run it locally against $KNN_INDEX first
```