README.md

---
title: Topic Feed Lab
emoji: 🧭
colorFrom: indigo
colorTo: green
sdk: docker
app_port: 7860
pinned: false
short_description: Follow and browse new Hub items by topic
---

# Topic Feed Lab

Follow new Hugging Face Hub models, datasets and Daily Papers by topic, or browse a topic to find what appeared this week.

## What it does

- **Live:** the newest items for the topics you pick, newest first, with a marker for what is new since your last visit.
- **Browse:** pick one or more topics, a time range, sources and a search term. Sort by topic score, newest, likes or
  downloads. A summary shows the counts per source and what else those items are about.
- **Confirm:** tick the topics that fit an item and press Confirm. After 3 confirmations for a topic, its threshold is
  fitted to your answers. Your choices stay in your browser.

## How it works

- A worker follows the Hub and Daily Papers, skips items without a real card, and scores each item once with
  [fastino/GLiNER2.5-Decide](https://huggingface.co/fastino/GLiNER2.5-Decide): one call gives a 0–1 score for each of
  48 catalogue topics. It is a small open classifier running on CPU, not an LLM.
- The scores are stored in the bucket `davanstrien/hub-firehose` (private for now), mounted in this Space at `/data`.
  A background worker process in this Space (cpu-upgrade, CPU only, lower priority than the page) watches the Hub for
  new and updated models and datasets plus HF Daily Papers, checks whether the card changed, and scores changed items
  with GLiNER2.5-Decide. The page loads the scores into memory and picks up new files every 20 s.
- The app is FastAPI served by Granian (Docker Space, one app process). It supervises exactly one ingest worker
  (a file lock enforces this). JSON API: `GET /api/catalogue`, `/api/status`, `/api/live`, `/api/events`,
  `/api/browse` (list params repeated, e.g. `?topics=robotics&topics=video`; `thresholds` is a JSON object;
  docs at `/api/docs`).
- Default thresholds (`cat48-v1.1.json`) are set so each topic tags about as many items as an LLM reference labelling
  did on a 750-item sample, with a floor of 0.1.

## Links and filters

- Every view has a shareable URL, e.g.
  `https://huggingface.co/spaces/davanstrien/topic-feed-lab?view=browse&topics=robotics&days=7&sources=dataset&sort=likes`
  (params: `view`, `topics` comma-separated, `items=all`, `days`, `sources`, `sort`, `q`, `how=closed`, `cardonly=1`).
  Your thresholds and confirmations stay in your browser and are not part of the link.
- **Card only:** repos with a README but no data files (datasets) or weight files (models) yet are hidden by default;
  tick "card-only" to include them. The worker records this at scoring time (`has_files`) and re-checks a card-only
  repo when it changes again, so it shows up once its files arrive. Items scored before this was added are "unknown"
  and always shown.

## Limitations

- Topics are zero-shot: some tags are wrong, and broad topics catch loosely related items. Confirm and adjust.
- Likes and downloads are a snapshot from scoring time.
- Papers have dates, not times; "today" means the latest paper day.
- Custom topics are not available yet (they need the model; planned for the worker).
- Private test Space.