Run on ZeroGPU: add @spaces.GPU entry point

DEPLOY.md

@@ -1,58 +1,66 @@
-# Deploying this to a Space
+# Deploying to LukeFP/Physh_Classification
 
-## 1. Create the Space
+The Space repo lives at `~/code/2026.7/Physh_Classification`.
 
-On huggingface.co → **New** → **Space**:
+## 1. Add the token secret
 
-- Owner `LukeFP`, name `physh-topic-classifier`
-- SDK **Gradio**, hardware **CPU basic** (free) — EmbeddingGemma-300m runs on
-  CPU in roughly a second per abstract, so a GPU buys little here
-- Visibility public or private, your call
+`google/embeddinggemma-300m` is gated. Accept the Gemma license while signed in,
+create a **read** token, then on the Space page: Settings → *Variables and
+secrets* → **New secret**, name `HF_TOKEN`, value the token. Without it the Space
+boots fine but the first classification fails with a 401.
 
-## 2. Add the token secret
+## 2. Hardware
 
-`google/embeddinggemma-300m` is gated. Accept the Gemma license while signed in,
-create a **read** token, then in the Space: Settings → *Variables and secrets* →
-**New secret**, name `HF_TOKEN`, value the token.
+On the free tier, Gradio Spaces run on **ZeroGPU**, which stops the container at
+startup unless it finds at least one `@spaces.GPU` function — the
+`No @spaces.GPU function detected during startup` error. `infer()` in `app.py`
+carries that decorator, so ZeroGPU is satisfied.
+
+Constraints ZeroGPU imposes, and how `app.py` meets them:
 
-If the model repo is private too, the same token covers it.
+| Constraint | Handling |
+|---|---|
+| `import spaces` must precede `import torch` | It is the first import in `app.py` |
+| Nothing may touch CUDA outside a `@GPU` function | Models load with `device="cpu"`; `.to(device)` happens inside `infer()` |
+| Return values cross a process boundary | `infer()` returns plain `list[float]`, never CUDA tensors |
+| One GPU allocation per call, with a duration budget | `@GPU(duration=60)`; the model is already resident, so only the encode runs |
+
+CPU basic (a PRO perk) also works with this code unchanged — `spaces` is an
+optional import and the device is chosen from `torch.cuda.is_available()`.
 
 ## 3. Push
 
 ```bash
-cd /Users/firstprinciplesextralaptop2/code/2026.7/physh-topic-classifier-space
-
-git init
-git remote add origin https://huggingface.co/spaces/LukeFP/physh-topic-classifier
-git add app.py requirements.txt README.md .gitignore DEPLOY.md test_local.py
-git commit -m "Gradio app for PhySH discipline + concept classification"
+cd ~/code/2026.7/Physh_Classification
 git push origin main
 ```
 
-If the Space was created with a README already, `git pull --rebase origin main`
-first — the frontmatter in this README is the one you want to keep, since it
-carries the `sdk_version` and `app_file` settings.
+The build takes a few minutes, most of it `pip install torch`.
 
-The build takes a few minutes, mostly `pip install torch`.
+## 4. First checks
 
-## 4. Check it
-
-Open the Space and run one of the built-in examples. Things to look at:
-
-- **Predictions look like noise / nothing clears the threshold.** Almost
+- **Predictions look like noise, or nothing clears the threshold.** Almost
   certainly the embedding prompt. Open *Advanced* and try the other two formats;
-  the one matching your training pipeline will give confident, coherent labels.
+  the one matching your training pipeline gives confident, coherent labels.
   Once you know which, set `DEFAULT_PROMPT` at the top of `app.py`.
-- **Error mentioning gated repo or 401.** `HF_TOKEN` is missing, wrong, or the
-  account behind it hasn't accepted the Gemma license.
-- **First request is slow.** Expected — EmbeddingGemma loads lazily on first use
-  so the Space boots fast. It's cached after that.
+  (`~/code/2026/embedding_title_abstract` likely has the answer.)
+- **Error mentioning a gated repo, or a 401.** `HF_TOKEN` is missing, wrong, or
+  the account behind it hasn't accepted the Gemma license.
+- **First request is slow, later ones fast.** Expected — EmbeddingGemma loads
+  lazily on first use so the Space boots quickly. Cached after that.
+
+## Updating later
+
+Retraining only needs a push to
+[`LukeFP/physh_topic_supervised_classifier`](https://huggingface.co/LukeFP/physh_topic_supervised_classifier);
+the Space picks up new weights on its next restart. Only change this repo if the
+*filenames* change — they're the constants at the top of `app.py`.
 
 ## Local smoke test
 
 Runs the real checkpoints through the full chain with a stubbed embedder, so it
-needs no token and no download:
+needs no token and no model download:
 
 ```bash
-python test_local.py
+PHYSH_WEIGHTS_DIR=~/code/2026.7/physh_topic_supervised_classifier python test_local.py
 ```

README.md

@@ -54,6 +54,11 @@ model page, then add a read token as a Space secret named `HF_TOKEN`
 (Settings → Variables and secrets). Without it the Space boots but the first
 classification fails.
 
+This Space runs on **ZeroGPU**: `infer()` carries the `@spaces.GPU` decorator,
+the models are loaded on CPU in the main process, and device placement happens
+inside the decorated function. The same code runs unchanged on CPU hardware —
+`spaces` is optional at import and `torch.cuda.is_available()` picks the device.
+
 ### Prompt format
 
 EmbeddingGemma prepends a task-specific prefix, and the prefix used here must
@@ -80,7 +85,7 @@ labelling:
 ```python
 from gradio_client import Client
 
-client = Client("LukeFP/physh-topic-classifier")
+client = Client("LukeFP/Physh_Classification")
 disciplines, concepts, summary = client.predict(
     "Title and abstract…", 0.5, "document — title: none | text: {}", 8,
     api_name="/classify",

app.py

@@ -1,5 +1,5 @@
 """
-PhySH topic classifier — Gradio Space.
+PhySH topic classifier — Gradio Space (ZeroGPU).
 
 Pipeline:  text ──EmbeddingGemma-300m──> 768-d vector
                       │
@@ -15,9 +15,24 @@ carry several disciplines and several concepts.
 
 from __future__ import annotations
 
-import functools
 import os
 
+# `spaces` must be imported before torch — it patches CUDA init so the main
+# process stays GPU-free until a @GPU function actually runs. ZeroGPU also scans
+# for at least one decorated function at startup and stops the container if it
+# finds none. The fallback keeps local runs and test_local.py working without it.
+try:
+    import spaces
+
+    GPU = spaces.GPU
+except ImportError:  # local development, or CPU hardware
+
+    def GPU(*dargs, **dkwargs):
+        if len(dargs) == 1 and callable(dargs[0]) and not dkwargs:
+            return dargs[0]
+        return lambda fn: fn
+
+
 import gradio as gr
 import torch
 import torch.nn as nn
@@ -32,8 +47,8 @@ DISCIPLINE_CKPT = "discipline_classifier_gemma_20260130_140842.pt"
 CONCEPT_CKPT = "concept_conditioned_gemma_20260130_140842.pt"
 EMBED_MODEL = "google/embeddinggemma-300m"
 
-# google/embeddinggemma-300m is a gated repo: set HF_TOKEN as a Space secret,
-# using a token from an account that has accepted the Gemma license.
+# google/embeddinggemma-300m is gated: set HF_TOKEN as a Space *secret*, from an
+# account that has accepted the Gemma license. Never commit the token itself.
 HF_TOKEN = os.environ.get("HF_TOKEN") or os.environ.get("HUGGINGFACE_HUB_TOKEN")
 
 # Set to a local directory to load the .pt files from disk instead of the Hub.
@@ -102,14 +117,18 @@ _CONDITION_ORDER = [d["discipline_id"] for d in CONCEPT_CKPT_DATA["discipline_la
 _HEAD_ORDER = [d["discipline_id"] for d in DISCIPLINE_CKPT_DATA["class_labels"]]
 _REMAP = torch.tensor([_HEAD_ORDER.index(i) for i in _CONDITION_ORDER], dtype=torch.long)
 
-
-@functools.lru_cache(maxsize=1)
-def get_embedder():
-    """Loaded on first request rather than at import, so the Space boots quickly
-    and a missing token surfaces as a readable error instead of a crashed app."""
+# EmbeddingGemma is loaded on CPU in the main process — under ZeroGPU nothing may
+# touch CUDA outside a @GPU function, and the fork inherits this copy for free.
+# A load failure is captured rather than raised so the Space still boots and can
+# report the reason in the UI instead of crash-looping.
+_EMBEDDER = None
+_EMBEDDER_ERROR: str | None = None
+try:
     from sentence_transformers import SentenceTransformer
 
-    return SentenceTransformer(EMBED_MODEL, token=HF_TOKEN)
+    _EMBEDDER = SentenceTransformer(EMBED_MODEL, token=HF_TOKEN, device="cpu")
+except Exception as exc:  # noqa: BLE001 — surfaced to the user verbatim
+    _EMBEDDER_ERROR = f"{type(exc).__name__}: {exc}"
 
 
 # --------------------------------------------------------------------------- #
@@ -117,30 +136,53 @@ def get_embedder():
 # --------------------------------------------------------------------------- #
 
 
-@torch.inference_mode()
+@GPU(duration=60)
+def infer(text: str, prompt_template: str) -> tuple[list[float], list[float]]:
+    """Embed and run both heads. Returns plain lists — ZeroGPU pickles the return
+    value across a process boundary, so nothing CUDA-resident may escape."""
+    if _EMBEDDER is None:
+        raise gr.Error(
+            "EmbeddingGemma failed to load. It is a gated model, so the Space needs "
+            "an HF_TOKEN secret from an account that has accepted the Gemma "
+            f"license.\n\n{_EMBEDDER_ERROR}"
+        )
+
+    device = "cuda" if torch.cuda.is_available() else "cpu"
+    embedder = _EMBEDDER.to(device)
+    discipline_model = DISCIPLINE_MODEL.to(device)
+    concept_model = CONCEPT_MODEL.to(device)
+    remap = _REMAP.to(device)
+
+    with torch.inference_mode():
+        vector = embedder.encode(
+            prompt_template.format(text),
+            prompt="",  # stop ST applying the model's own default prefix on top
+            convert_to_numpy=True,
+        )
+        embedding = torch.as_tensor(vector, dtype=torch.float32, device=device).unsqueeze(0)
+
+        discipline_probs = torch.sigmoid(discipline_model(embedding))[0]
+        conditioned = torch.cat([embedding, discipline_probs[remap].unsqueeze(0)], dim=1)
+        concept_probs = torch.sigmoid(concept_model(conditioned))[0]
+
+    return discipline_probs.float().cpu().tolist(), concept_probs.float().cpu().tolist()
+
+
 def classify(text: str, threshold: float, prompt_choice: str, top_k: int):
     text = (text or "").strip()
     if not text:
         return {}, {}, "Paste some text — a title and abstract work best."
 
-    prompt = PROMPT_TEMPLATES.get(prompt_choice, PROMPT_TEMPLATES[DEFAULT_PROMPT])
-    # prompt="" stops sentence-transformers from also applying the model's own
-    # default prefix on top of the one built here.
-    vector = get_embedder().encode(prompt.format(text), prompt="", convert_to_numpy=True)
-    embedding = torch.as_tensor(vector, dtype=torch.float32).unsqueeze(0)
-
-    discipline_probs = torch.sigmoid(DISCIPLINE_MODEL(embedding))[0]
-    conditioned = torch.cat([embedding, discipline_probs[_REMAP].unsqueeze(0)], dim=1)
-    concept_probs = torch.sigmoid(CONCEPT_MODEL(conditioned))[0]
+    template = PROMPT_TEMPLATES.get(prompt_choice, PROMPT_TEMPLATES[DEFAULT_PROMPT])
+    discipline_scores, concept_scores = infer(text, template)
 
-    disciplines = {DISCIPLINE_LABELS[i]: float(p) for i, p in enumerate(discipline_probs)}
-    concepts = {CONCEPT_LABELS[i]: float(p) for i, p in enumerate(concept_probs)}
+    disciplines = dict(zip(DISCIPLINE_LABELS, discipline_scores))
+    concepts = dict(zip(CONCEPT_LABELS, concept_scores))
 
-    summary = _summarize(disciplines, concepts, threshold)
     return (
         dict(sorted(disciplines.items(), key=lambda kv: -kv[1])[:top_k]),
         dict(sorted(concepts.items(), key=lambda kv: -kv[1])[:top_k]),
-        summary,
+        _summarize(disciplines, concepts, threshold),
     )
 
 

requirements.txt

@@ -1,13 +1,12 @@
 gradio==6.28.0
 
+# ZeroGPU: provides the @spaces.GPU decorator the Space is scanned for at startup.
+spaces
+
 # EmbeddingGemma needs sentence-transformers >= 5.0 / transformers >= 4.56,
-# and does not support float16 activations (float32 on CPU is the default here).
+# and does not support float16 activations — this app keeps it in float32.
 sentence-transformers>=5.0
 transformers>=4.56
 huggingface_hub>=0.34
 
-# PyPI's torch wheel bundles CUDA (~3 GB) and is wasted on a CPU Space. If build
-# times or disk become a problem, replace the line below with these two:
-#   --extra-index-url https://download.pytorch.org/whl/cpu
-#   torch>=2.4
 torch>=2.4

test_local.py

@@ -1,42 +1,71 @@
 """Smoke test: loads the real checkpoints, stubs out EmbeddingGemma, and runs the
-full chain. Run with PHYSH_WEIGHTS_DIR pointing at the cloned model repo."""
-import os, sys, numpy as np, torch
+full chain on CPU. Needs no HF token and downloads nothing.
+
+    PHYSH_WEIGHTS_DIR=../physh_topic_supervised_classifier python test_local.py
+"""
+
+import os
 
 os.environ.setdefault(
     "PHYSH_WEIGHTS_DIR",
     os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "physh_topic_supervised_classifier"),
 )
+
+import numpy as np
+import torch
+
 import app
 
+
 class FakeEmbedder:
+    """Stands in for SentenceTransformer: deterministic unit-norm vectors."""
+
+    def to(self, device):
+        return self
+
     def encode(self, text, prompt=None, convert_to_numpy=True):
         rng = np.random.default_rng(abs(hash(text)) % (2**32))
         v = rng.normal(size=768).astype("float32")
         return v / np.linalg.norm(v)  # EmbeddingGemma returns L2-normalised vectors
 
-app.get_embedder.cache_clear()
-app.get_embedder = lambda: FakeEmbedder()
+
+app._EMBEDDER = FakeEmbedder()
 
 print("disciplines:", len(app.DISCIPLINE_LABELS), "concepts:", len(app.CONCEPT_LABELS))
 print("remap is identity:", torch.equal(app._REMAP, torch.arange(18)))
-print("discipline net:", app.DISCIPLINE_MODEL.network)
-print("concept  in_features:", app.CONCEPT_MODEL.network[0].in_features)
+print("concept head in_features:", app.CONCEPT_MODEL.network[0].in_features)
+import sys
+print("real `spaces` package in use:", "spaces" in sys.modules)
 
 d, c, summary = app.classify(app.EXAMPLES[0], 0.5, app.DEFAULT_PROMPT, 8)
 assert len(d) == 8 and len(c) == 8, (len(d), len(c))
 assert all(0.0 <= v <= 1.0 for v in {**d, **c}.values())
-print("\ntop disciplines:", [f"{k} {v:.3f}" for k, v in d.items()][:4])
-print("top concepts:   ", [f"{k} {v:.3f}" for k, v in c.items()][:4])
-print("\n--- summary ---\n" + summary)
+print("\ntop disciplines:", [f"{k} {v:.3f}" for k, v in d.items()][:3])
+print("top concepts:   ", [f"{k} {v:.3f}" for k, v in c.items()][:3])
 
-# determinism (dropout must be off) and empty input
+# dropout must be off, so repeat calls agree
 d2, _, _ = app.classify(app.EXAMPLES[0], 0.5, app.DEFAULT_PROMPT, 8)
 assert d == d2, "eval() mode not applied — outputs are not deterministic"
-empty = app.classify("   ", 0.5, app.DEFAULT_PROMPT, 8)
-assert empty[0] == {} and "Paste" in empty[2]
+
+# empty input, and every prompt format
+assert app.classify("   ", 0.5, app.DEFAULT_PROMPT, 8)[0] == {}
 for p in app.PROMPT_TEMPLATES:
     app.classify("test abstract", 0.5, p, 5)
-print("\nOK: deterministic, empty input handled, all 3 prompt formats run.")
 
-app.demo  # Blocks built at import; confirm the UI graph is constructed
-print("gradio Blocks built OK")
+# the GPU-decorated entry point returns plain lists (ZeroGPU pickles them)
+dl, cl = app.infer("test", app.PROMPT_TEMPLATES[app.DEFAULT_PROMPT])
+assert isinstance(dl, list) and isinstance(cl, list) and len(dl) == 18 and len(cl) == 186
+assert all(isinstance(x, float) for x in dl)
+
+# a missing embedder must surface as a readable error, not an AttributeError
+app._EMBEDDER, app._EMBEDDER_ERROR = None, "401 gated repo"
+try:
+    app.infer("test", "{}")
+    raise SystemExit("FAIL: expected a gr.Error when the embedder is missing")
+except Exception as exc:
+    assert "HF_TOKEN" in str(exc), exc
+app._EMBEDDER = FakeEmbedder()
+
+print("\nOK: deterministic, empty input handled, all prompt formats run,")
+print("    infer() returns picklable lists, missing-token error is readable.")
+print("gradio Blocks built OK:", app.demo is not None)