feat(search): relevance reranking with Laya, and restore files the last commit clobbered

Adds the reranker, and puts back what 4dba37a overwrote.

4dba37a was a mistake: it synced the whole of backend/space/ over the Space,
and this repo's copy of that folder is a STALE FORK. It reverted app.py's
_KEY_HEADERS alias (the x-respite-key / x-ael-key fix), its _is_protected fix,
the raised request_timeout in searxng_settings.yml, and the explicit
mojeek/marginalia/rightdao/mwmbl/presearch engine list. Restored here from
17b0658; this commit touches only the three files it meant to.

The reranker itself:
- rerank.py scores each (query, passage) pair with convaiinnovations/laya, a
non-autoregressive System 1 decision model (421M English / 322M multilingual,
Apache-2.0), using a `noul` question for the calibrated P(yes) that the
passage contains what the query needs.
- The race only ever asked "is this set big enough", so the winner was whoever
answered first and a small index won on speed alone. "what is the capital of
Peru" came back as NPR transcripts about mummy lice and an article about
mosquitoes in Delhi: eight results, none answering it, degraded: false. A
count check cannot see that and neither can lexical overlap, since
"Viceroyalty of Peru" really does contain both words.
- Strictly fail-open. Model missing, disabled, scoring exception, over budget,
or a model that rejects everything: the original set comes back untouched.
A reranker is the only part of search that can make results worse, so every
failure mode has to be a no-op.
- POST /respite/v2/searx/rerank, under the prefix the key gate already covers,
so it needs no second authentication to get right.
- GET /respite/v2/searx/rerank/recent serves the recent score distribution,
because AEL_RERANK_DROP is a guess that should be chosen from real traffic.

🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>

requirements.txt

@@ -10,3 +10,9 @@ ddgs
 transformers>=4.51.0
 accelerate>=1.6.0
 httpx
+
+# Relevance reranker for search results (backend/space/rerank.py). Apache-2.0,
+# non-autoregressive, 421M (english) / 322M (multilingual), one forward pass per
+# document. rerank.py degrades to "no opinion" if this import fails, so a
+# failed install costs search quality, not search itself.
+laya>=0.1.0

rerank.py

@@ -0,0 +1,231 @@
+"""Relevance reranking for search results, using Laya (convaiinnovations).
+
+Why this exists
+---------------
+The SearXNG bridge races several engines and hands back the first set that
+looks big enough. Nothing in that path asks whether the results are *about* the
+query, so the fastest engine wins — and a small independent index is fast.
+Measured on production, 2026-10-02, from the route's own `source` field:
+
+    "what is the capital of Peru"            -> mwmbl -> NPR transcripts about
+        mummy lice, an article about mosquitoes in Delhi, "The Capital Cycle".
+        Eight results. None of them answers the question.
+    "Peru capital city"                     -> mwmbl -> a martial arts studio
+        in Wellington, an NScale railroad modellers' club.
+    "Donald Trump AI regulations policies"  -> bing  -> "Donald Trump -
+        Wikipedia", AP's Trump hub, Reuters' Trump hub.
+
+None of that is an outage, so nothing raised an alarm: HTTP 200, `degraded:
+false`, a full result page. It is the failure mode where the search path is
+technically healthy and completely useless, and a count-based "is the set big
+enough" check cannot see it — a lexical overlap check cannot either, because
+"Viceroyalty of Peru" really does contain the words "Peru" and "capital" and
+still does not answer the question.
+
+A cross-encoder can, because it reads the query and the passage together and
+answers "does this passage answer this question" rather than "do these strings
+overlap". Laya is the right size for that: a non-autoregressive System 1
+decision model, 421M params in English (ModernBERT-large) and 322M for 100+
+languages (mmBERT-base), one forward pass per document, Apache-2.0, calibrated
+probabilities via RLCD. It never generates text, so there is nothing to parse
+and nothing to hallucinate.
+
+Design constraints, all of them learned the hard way
+----------------------------------------------------
+* **Fail open, always.** If the model is missing, slow, or the import breaks,
+  the caller gets its results back untouched. Search must not depend on a
+  component that is described as "kind of dumb" and can be swapped out.
+* **Never return an empty page.** If the model rejects every result, that is
+  not a confident judgement, it is a model that did not understand the query —
+  so the original set is returned and `starved` is set. That flag is the signal
+  to retune DROP_BELOW, not a silent success.
+  An earlier version of this file had a floor of 3 surviving results, on the
+  theory that a short page is a bad page. That was wrong in the exact case
+  this module exists for: the mwmbl result set for "what is the capital of Peru"
+  is eight results of which perhaps two are worth showing, and a floor of 3
+  throws those two away and serves the eight instead. A short honest page beats
+  a full wrong one. The only floor that is genuinely wrong is zero.
+* **Log every score.** The threshold is not a guess to be shipped and admired;
+  it is a number to be chosen from the distribution below. RECENT keeps the
+  last N (query, scores) so the threshold can be tuned from real traffic.
+* **Deadline-bounded.** Reranking is an enhancement, never a reason to make a
+  user wait: the caller passes a budget and this module stops at it.
+"""
+
+import os
+import threading
+import time
+from collections import deque
+
+# ── configuration ───────────────────────────────────────────────────────────
+# Off by default so a bad deploy cannot take search down; the route treats a
+# disabled reranker as "no opinion" and returns what SearXNG gave it.
+ENABLED = os.environ.get("AEL_RERANK", "1") not in ("0", "false", "no")
+
+# "english" is 421M ModernBERT-large, 512 tokens, the strongest.
+# "multilingual" is 322M mmBERT-base, ~2x faster, 100+ languages. The route
+# sends whichever it wants; the default here is the stronger English one
+# because Ael's corpus is overwhelmingly English, and non-English queries are
+# still scored correctly by it far more often than not.
+MODEL = os.environ.get("AEL_RERANK_MODEL", "english")
+
+# Drop a result when the model thinks it is more likely NOT to answer the query
+# than to answer it. Deliberately conservative: 0.5 is the neutral point of a
+# calibrated probability, so this only removes what the model actively rejects.
+DROP_BELOW = float(os.environ.get("AEL_RERANK_DROP", "0.5"))
+
+# Never return an empty page. See the starvation note above — this is a floor of
+# one, not a target: a short page is fine, an empty one is not.
+MIN_RESULTS = 1
+
+# Recent (query, scores) for threshold tuning. Bounded so a long-lived Space
+# cannot grow without limit.
+RECENT = deque(maxlen=int(os.environ.get("AEL_RERANK_LOG", "200")))
+
+_router = None
+_router_lock = threading.Lock()
+_load_error = None
+
+
+def _get_router():
+    """Load the checkpoint once, lazily. Returns None if it cannot be loaded.
+
+    Deliberately never raises: a Space that cannot afford the model still
+    serves search, and the route needs to be able to ask "any opinion?"
+    without handling an exception.
+    """
+    global _router, _load_error
+    if _router is not None or _load_error is not None:
+        return _router
+    with _router_lock:
+        if _router is not None or _load_error is not None:
+            return _router
+        try:
+            from laya import Router
+
+            # max_loaded=1: one checkpoint resident, which is all this uses.
+            _router = Router(max_loaded=1)
+            _router.preload([MODEL])
+        except Exception as e:  # noqa: BLE001 - reported, never raised
+            _load_error = f"{type(e).__name__}"
+            print(f"[rerank] disabled: cannot load laya ({_load_error}): {e}", flush=True)
+    return _router
+
+
+def status():
+    """For the /status route: is this thing actually working right now."""
+    router = _get_router()
+    return {
+        "enabled": ENABLED,
+        "ready": router is not None,
+        "model": MODEL,
+        "drop_below": DROP_BELOW,
+        "min_results": MIN_RESULTS,
+        "load_error": _load_error,
+        "samples_logged": len(RECENT),
+    }
+
+
+def recent_scores(limit=20):
+    """Recent (query, scores) pairs, newest first — the threshold-tuning data."""
+    items = list(RECENT)[-limit:][::-1]
+    return [{"q": q, "scores": s} for q, s in items]
+
+
+def _question(query):
+    """The typed question. `noul` returns the calibrated probability of YES.
+
+    The query is repeated in the instructions rather than passed as context
+    because that is the documented shape: state is the passage, questions carry
+    the ask. Asking in the imperative ("does this passage contain the answer")
+    is what makes it a relevance judgement and not a topic match.
+    """
+    return {
+        "relevant": {
+            "type": "noul",
+            "instructions": (
+                "Does this passage contain the information needed to answer the "
+                "web search query below?\n"
+                f"Query: {query}\n"
+                "Answer no if the passage is about a related topic but does not "
+                "actually address the query, and no if it is a navigation page, "
+                "a category listing, or a site homepage."
+            ),
+        }
+    }
+
+
+def _passage(result):
+    """The text the model judges. Title carries most of the topical signal;
+    snippet is what we have of the page without fetching it."""
+    title = str(result.get("title") or "").strip()
+    snippet = str(result.get("snippet") or "").strip()
+    return (title + "\n" + snippet).strip()
+
+
+def rerank(query, results, budget_ms=800):
+    """Score, reorder and optionally drop `results` for `query`.
+
+    Returns a dict; never raises. On any problem `results` comes back in its
+    original order with `ok: false`, because a reranker that degrades search
+    is worse than no reranker.
+    """
+    out = {
+        "ok": False,
+        "results": results,
+        "model": MODEL,
+        "drop_below": DROP_BELOW,
+        "dropped": 0,
+        "starved": False,
+        "scores": [],
+    }
+    if not ENABLED or not results:
+        return out
+    router = _get_router()
+    if router is None:
+        return out
+
+    t0 = time.time()
+    scored = []
+    for i, r in enumerate(results):
+        passage = _passage(r)
+        if not passage:
+            # No text to judge. Keep it, unscored, and let the ordering stand.
+            scored.append((None, i, r))
+            continue
+        try:
+            res = router.predict(passage, _question(query))
+            p = float(res["answers"]["relevant"]["noul"])
+        except Exception as e:  # noqa: BLE001
+            print(f"[rerank] score failed for result {i}: {type(e).__name__}: {e}", flush=True)
+            return dict(out, results=results)  # fail open, whole set untouched
+        if (time.time() - t0) * 1000 > budget_ms:
+            # Over budget: return what we have not yet reordered, in order.
+            print(f"[rerank] over budget ({budget_ms}ms) after {i} results", flush=True)
+            return dict(out, results=results)
+        scored.append((p, i, r))
+
+    if not any(p is not None for p, _, _ in scored):
+        return out
+
+    # Highest score first. Unscored results keep their relative position at the
+    # end rather than being dropped: we have no opinion about them.
+    scored.sort(key=lambda t: (t[0] is None, -(t[0] if t[0] is not None else 0.0), t[1]))
+    kept = [r for p, _, r in scored if p is None or p >= DROP_BELOW]
+    dropped = len(scored) - len(kept)
+
+    out["scores"] = [
+        {"title": str(r.get("title") or "")[:80], "score": p}
+        for p, _, r in scored
+        if p is not None
+    ]
+    RECENT.append((query[:120], [p for p, _, _ in scored if p is not None]))
+
+    if dropped and len(kept) < MIN_RESULTS:
+        # The model rejected everything, which is a model that did not
+        # understand the query rather than a set with nothing in it. Say so
+        # instead of pretending.
+        return dict(out, results=results, starved=True)
+
+    out.update(ok=True, results=kept, dropped=dropped)
+    return out

searx_bridge.py

@@ -2,17 +2,26 @@
 container (userspace venv, no root) and proxies queries to it.
 
 Mounted under /respite/v2/searx/*:
-  /status  → bootstrap state of the runtime
+  /status  → bootstrap state of the runtime, and of the reranker
   /search  → proxy ?q=... queries to SearXNG's JSON API
+  /rerank  → score/reorder/drop a result set for a query (see rerank.py)
 
 Engine selection comes from searxng_settings.yml (DDG + Google enabled);
 the frontend picks engines per request via the standard `engines` param.
+
+/rerank lives under the same prefix as /search on purpose: the Space's key gate
+guards /respite/v2/searx/*, so the reranker inherits that authentication rather
+than needing a second one to be got right.
 """
 
+import asyncio
+import json
+
 import httpx
 from fastapi import Request
 from fastapi.responses import JSONResponse
 
+import rerank
 import searx_runtime
 
 
@@ -24,6 +33,7 @@ def register_routes(fa_app):
             "installed": searx_runtime.searxng_ready(),
             "boot_error": searx_runtime.boot_error(),
             "port": searx_runtime.SEARXNG_PORT,
+            "reranker": rerank.status(),
         })
 
     @fa_app.get("/respite/v2/searx/search")
@@ -42,3 +52,35 @@ def register_routes(fa_app):
                 return JSONResponse(r.json(), status_code=r.status_code)
         except Exception as e:
             return JSONResponse({"error": str(e)}, status_code=502)
+
+    @fa_app.post("/respite/v2/searx/rerank")
+    async def _rerank(request: Request):
+        """Rerank a result set. Always answers 200 with the results in some
+        order — the client is a search path, and an exception here would look
+        like an outage to whoever is trying to read the page."""
+        try:
+            body = await request.body()
+            payload = json.loads(body or b"{}")
+        except Exception as e:
+            return JSONResponse({"ok": False, "error": f"bad request: {e}", "results": []})
+
+        q = str(payload.get("q") or "").strip()
+        results = payload.get("results")
+        if not q or not isinstance(results, list) or not results:
+            return JSONResponse({"ok": False, "error": "q and results are required", "results": results or []})
+
+        budget_ms = int(payload.get("budget_ms") or 800)
+        # rerank() is synchronous and CPU-bound (torch), so it must not run on
+        # the event loop — that would stall every other Space request, audio
+        # generation included, for as long as the model takes.
+        try:
+            out = await asyncio.to_thread(rerank.rerank, q, results, budget_ms)
+        except Exception as e:  # noqa: BLE001
+            return JSONResponse({"ok": False, "error": f"rerank failed: {e}", "results": results})
+        return JSONResponse(out)
+
+    @fa_app.get("/respite/v2/searx/rerank/recent")
+    def _rerank_recent(limit: int = 20):
+        """Recent relevance scores, for choosing DROP_BELOW from real traffic
+        rather than from a guess."""
+        return JSONResponse({"scores": rerank.recent_scores(limit)})