Document offline usage

README.md

@@ -56,6 +56,50 @@ print(classify([
 
 Classify in batches of 16–32 articles; `max_len=256` is about twice as fast at a small quality cost (see Results).
 
+## Download once, use without Hugging Face
+
+1. Download the files (needs access to this private repo once), with any of:
+   - CLI: `hf auth login`, then `hf download maxafanasiev/news-topic-classifier-e5-base --local-dir ./news-topic-classifier-e5-base`
+   - git: `git lfs install && git clone https://huggingface.co/maxafanasiev/news-topic-classifier-e5-base` (use a token as the password)
+   - browser: the "Files and versions" tab, download every file into one folder
+2. Keep these files together: `model.safetensors`, `config.json`, `tokenizer.json`, `tokenizer_config.json`, `thresholds.json`.
+3. Load from the folder; `HF_HUB_OFFLINE=1` makes sure nothing is fetched from the internet:
+
+```python
+import json
+import os
+from pathlib import Path
+
+os.environ["HF_HUB_OFFLINE"] = "1"
+
+import torch
+from transformers import AutoModelForSequenceClassification, AutoTokenizer
+
+MODEL_DIR = Path("./news-topic-classifier-e5-base")
+PREFIX = "passage: "
+PARENTS = {"Theft": "Crime", "Assault": "Crime"}
+
+tokenizer = AutoTokenizer.from_pretrained(MODEL_DIR)
+model = AutoModelForSequenceClassification.from_pretrained(MODEL_DIR).eval()
+thresholds = json.loads((MODEL_DIR / "thresholds.json").read_text())
+categories = list(thresholds)
+
+
+@torch.no_grad()
+def classify(articles: list[dict], max_len: int = 512) -> list[list[str]]:
+    texts = [f"{PREFIX}{a['title']}\n{a['content']}" for a in articles]
+    batch = tokenizer(texts, truncation=True, max_length=max_len, padding=True, return_tensors="pt")
+    probs = torch.sigmoid(model(**batch).logits)
+    results = []
+    for row in probs:
+        labels = [c for c, p in zip(categories, row.tolist()) if p >= thresholds[c]]
+        labels += [PARENTS[c] for c in labels if c in PARENTS and PARENTS[c] not in labels]
+        results.append(labels or ["Other"])
+    return results
+```
+
+Requirements: `torch` and `transformers` (no `huggingface_hub` login needed at this point).
+
 ## Results (gold set, CPU Apple M4 Pro, fp32)
 
 | Max tokens | micro-F1 (random gold, 163) | exact set match | CPU latency |