Release armiger-zero-cue-catalog v2.0.0 standalone modular plugin

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+validated_zero_cue_catalog.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,3 +1,95 @@
 ---
 license: apache-2.0
+library_name: generic
+tags:
+  - norvitech
+  - armiger
+  - armiger-ultra
+  - armiger-plugin
+  - bitagent
+  - bitmagnet
+  - zero-cue
+  - entity-resolution
+  - text-classification
+  - sonarr
+  - radarr
+pipeline_tag: text-classification
 ---
+
+<h1 align="center">Armiger Plugin #3 — Zero-Cue Catalog &amp; Structural Disambiguator (<code>armiger-zero-cue-catalog</code>)</h1>
+
+<p align="center">
+  <strong>Resolves bare-title ("zero-cue") torrents lacking season/episode syntax via an offline 926,785-key normalized media catalog and structural release rules, lifting Movie-vs-TV accuracy to <code>99.46%</code> and 11-class accuracy to <code>98.48%</code>.</strong><br>
+  Available standalone here and pre-bundled inside <a href="https://huggingface.co/NorviTech/armiger-ultra"><code>NorviTech/armiger-ultra</code></a>.
+</p>
+
+<p align="center">
+  <a href="https://huggingface.co/NorviTech/armiger-ultra"><img alt="Bundled in Armiger Ultra" src="https://img.shields.io/badge/Bundled_in-Armiger_Ultra-FD8024.svg"></a>
+  <a href="https://huggingface.co/NorviTech/armiger"><img alt="Compatible with Armiger" src="https://img.shields.io/badge/Compatible_with-Armiger_CPU-0284C7.svg"></a>
+  <img alt="Licence Apache-2.0" src="https://img.shields.io/badge/licence-Apache--2.0-blue.svg">
+</p>
+
+## Why the Zero-Cue Catalog Plugin Exists
+
+In real-world BitTorrent/DHT indexing, **2% to 4%** of movie, TV, and anime torrents contain **zero structural cues** in either their title or file tree:
+- `The Riddle (2007) [1080p] [WEB-DL].mkv` (a standalone film vs. a 1-file TV special)
+- `[SubGroup] Bare Japanese/English Title [1080p].mkv` (a theatrical anime film vs. a TV OVA/series without an episode number)
+- `Director Name - Movies 1-6 Collection (1979-1998) 1080p` (a multi-file movie pack whose 6 `.mkv` files look identical in file-count to a 6-episode TV mini-series)
+
+Even a 70B+ parameter cloud LLM must guess when given only a bare title unless it can consult a media catalog. `NorviTech/armiger-zero-cue-catalog` pairs deterministic structural guards (for multi-movie packs, visual novels, console ROMs, and educational courses) with an offline, O(1) hash-table lookup across **926,785 normalized keys** (`472,038` `(title, year)` pairs, `257,008` anime title stems, and `197,739` unambiguous long titles).
+
+## Measured Accuracy Lift Across All 6 Evaluation Cohorts
+
+| Evaluation Cohort | Neural Backbone Only (`Armiger Ultra`) | + Structural Guards | **+ `armiger-zero-cue-catalog` (`Armiger Ultra`)** | **+ `armiger-zero-cue-catalog` (`Armiger 8.3 MB`)** |
+| :--- | :---: | :---: | :---: | :---: |
+| **Gold Standard (`N=2,835`)** | `97.45%` | `97.81%` | **`98.48%`** (`99.46%` Movie-vs-TV) | **`97.92%`** (`99.15%` Movie-vs-TV) |
+| **Fresh Torrents (`N=1,354`)** | `95.79%` | `95.94%` | **`96.60%`** | **`96.38%`** |
+| **Hard Edge-Cases (`N=1,025`)** | `95.71%` | `96.10%` | **`97.17%`** | **`96.20%`** |
+| **Validation Holdout (`N=2,237`)** | `96.92%` | `97.05%` | **`97.32%`** | **`96.51%`** |
+| **Specialist Torrents (`N=1,837`)** | `93.85%` | `94.18%` | **`98.26%`** | **`97.82%`** |
+| **Multilingual Torrents (`N=2,828`)** | `95.44%` | `95.69%` | **`98.02%`** | **`97.81%`** |
+
+## Quickstart
+
+```python
+from zero_cue_plugin import ZeroCueCatalogPlugin
+
+plugin = ZeroCueCatalogPlugin("validated_zero_cue_catalog.json")
+res = plugin.resolve(
+    raw_name="The Riddle (2007) [1080p] [WEB-DL]",
+    probs_category=[0.46, 0.51, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.03],
+    p_adult=0.01,
+    p_anime=0.02,
+)
+print(res)
+# -> {'category_index': 0, 'category': 'movie', 'resolution_source': 'zero_cue_title_year'}
+```
+
+## Part of the Armiger Modular Plugin Suite
+
+| Plugin Repository | Bundled in `Armiger Ultra`? | Footprint | Core Capability & Measured Impact |
+| :--- | :---: | :---: | :--- |
+| **[`NorviTech/armiger-swarm-survival`](https://huggingface.co/NorviTech/armiger-swarm-survival)** | **Yes** (`swarm_adapter.pt`, `swarm_survival.py`) | `163 KB` | **SwarmSurvivalNet v2 & Honeypot Residual Adapter:** **`96.83%`** Swarm Liveliness AUROC (vs. `53.04%` raw seeders), **`100.0%`** fake-seeder malware trap recall, **`0%`** content-category flip guarantee. |
+| **[`NorviTech/armiger-swarm-priors`](https://huggingface.co/NorviTech/armiger-swarm-priors)** | **Yes** (`swarm_priors.py`) | `29.4 KB` | **Empirical Bayes Swarm & Release-Group Priors:** 4-pathway tracker scrape prior engine, mandatory modality dropout (`p=0.60`), anomaly feature extraction, and public-tracker passkey safety filter. |
+| **[`NorviTech/armiger-zero-cue-catalog`](https://huggingface.co/NorviTech/armiger-zero-cue-catalog)** | **Yes** (`zero_cue_plugin.py`, `validated_zero_cue_catalog.json`) | `26.8 MB` | **Zero-Cue Catalog & Structural Disambiguator:** Offline 926,785-key normalized `TMDB`/`TVDB`/`AniDB`/`Manami` lookup + structural pack/ROM/VN/course rules; lifts Movie-vs-TV accuracy to **`99.46%`** and 11-class accuracy to **`98.48%`**. |
+| **[`NorviTech/armiger-audio-guard`](https://huggingface.co/NorviTech/armiger-audio-guard)** | **Yes** (`audio_language_guard.py`) | `4.2 KB` | **Cross-Head Audio Language & Anime Dub Guard:** Couples `title_language` (`P(Foreign Title)`) with `lang` head to disambiguate `.en.` prepositions, `<Foreign>.ENG.Sub`, regional dubs, and `MULTi (FR EN)` tags; reaches **`98.72%`** Gold Standard and **`99.88%`** Hard Edge-Case audio accuracy. |
+| **[`NorviTech/armiger-calibration-dht`](https://huggingface.co/NorviTech/armiger-calibration-dht)** | **Yes** (`calibration.json`, `calibration_and_dht.py`) | `8.5 KB` | **Conformal Calibration & BEP-44 DHT Updater:** Per-head temperature scaling ($T_h$), Menon prior adjustments, Mondrian conformal safety gates (`99.95%` junk balanced accuracy), and `Ed25519`-signed `BEP-44` decentralized DHT hot-reload engine. |
+
+---
+
+<p align="center">
+  <a href="https://norvitech.com"><img alt="Part of the NorviTech Suite — open-source apps for the Linux desktop and the self-hosted stack" src="https://norvitech.com/assets/banner.svg" width="640"></a>
+</p>
+
+<p align="center">
+  <a href="https://github.com/spencercnorton/helios">Helios</a> ·
+  <a href="https://github.com/spencercnorton/bitagent">BitAgent</a> ·
+  <a href="https://github.com/spencercnorton/xnote">XNote</a> ·
+  <a href="https://github.com/spencercnorton/xnote-placement">XNote Placement</a> ·
+  <a href="https://github.com/spencercnorton/snipsnap">SnipSnap</a> ·
+  <a href="https://github.com/spencercnorton/conductor">Conductor</a> ·
+  <a href="https://github.com/spencercnorton/norvi-os">NorviOS</a> ·
+  <a href="https://github.com/spencercnorton/indigo">Indigo</a> ·
+  <a href="https://github.com/spencercnorton/roadtrack">Road Track</a> ·
+  <a href="https://norvitech.com">norvitech.com</a>
+</p>

config.json

@@ -0,0 +1,28 @@
+{
+  "plugin_name": "armiger-zero-cue-catalog",
+  "version": "2.0.0",
+  "part_of_suite": "NorviTech/armiger-ultra",
+  "architecture": "offline-normalized-catalog-plus-structural-disambiguator",
+  "catalog_counts": {
+    "title_year_map": 472038,
+    "anime_map": 257008,
+    "title_only_map": 200365,
+    "total_keys": 929411
+  },
+  "benchmarks": {
+    "gold_standard_11class_accuracy_ultra_pct": 98.48,
+    "gold_standard_11class_accuracy_armiger_pct": 97.92,
+    "gold_standard_movie_vs_tv_accuracy_ultra_pct": 99.46,
+    "gold_standard_catalog_matching_rate_ultra_pct": 98.69
+  },
+  "artifacts": {
+    "zero_cue_plugin.py": {
+      "size_bytes": 10339,
+      "sha256": "9a4eb5261153ba8c5d53a79638e6ef71263faf4effd0f17e6b4ac905f7f14e79"
+    },
+    "validated_zero_cue_catalog.json": {
+      "size_bytes": 36004794,
+      "sha256": "e30f10ee99b89798b879a7c305099005ac48fc8fa8febebcaf10be154930b999"
+    }
+  }
+}

validated_zero_cue_catalog.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e30f10ee99b89798b879a7c305099005ac48fc8fa8febebcaf10be154930b999
+size 36004794

zero_cue_plugin.py

@@ -0,0 +1,252 @@
+"""Zero-Cue Catalog Lookup & Structural Disambiguation Plugin for Armiger & Armiger Ultra.
+
+Resolves bare-title ("zero-cue") torrent releases that omit explicit season/episode syntax
+(e.g. `Title (2024) [1080p].mkv`) and disambiguates multi-movie packs, visual novels,
+console ROM images, and educational courses before/after neural classification.
+"""
+from __future__ import annotations
+
+import json
+import pathlib
+import re
+import unicodedata
+from typing import Any
+
+CATS = [
+    "movie", "tv_show", "xxx", "music", "comic",
+    "game", "audiobook", "software", "ebook", "course", "unknown",
+]
+IDX_MOVIE = 0
+IDX_TV = 1
+IDX_XXX = 2
+IDX_MUSIC = 3
+IDX_COMIC = 4
+IDX_GAME = 5
+IDX_AUDIOBOOK = 6
+IDX_SOFTWARE = 7
+IDX_EBOOK = 8
+IDX_COURSE = 9
+
+EXPLICIT_TV_RE = re.compile(
+    r"(?i)(?:\bS\d{1,2}E\d{1,3}\b|\bS\d{1,2}\s*-\s*S\d{1,2}\b|\bSeason\s*\d+\b|\bStagione\s*\d+\b|\bSaison\s*\d+\b|"
+    r"\bTemporada\s*\d+\b|\bStaffel\s*\d+\b|\bComplete\s+Series\b|\bTV\s*Series\b|\bMini\s*Series\b|\bTVRip\b|"
+    r"\bEpisode\s+[1-9]\d*\b|\bEp\.?\s*[1-9]\d*\b|\bEpiz[oó]da\b|\bЭпизод\s*\d+\b|第[一二三四五六七八九十\d]+[季集]|【[^】]*日剧】|"
+    r"\bMyTVSuper\b|\bMOTOGP\b|\bFame\s+MMA\b|\bNJPW\b|\bUFC\s+\d+\b|\bFull[\s\._]Event\b|\bPPV\b|"
+    r"\b(?:One\s+Piece|Naruto|Bleach|Boruto)\s+\d{2,4}\b|\b1-\d{2}\s*\+\s*Movie\b|\btemp\.\d+\.completa\b|"
+    r"\b\d{2}\s*-\s*\d{2,3}\s+VOSTFR\b|\s-\s\d{2,3}(?:v\d)?\s*(?:\[|\(|\b1080p\b|\b720p\b))"
+)
+
+EXPLICIT_MOVIE_PACK_RE = re.compile(
+    r"(?i)(?:\bMovies?\s*\d{1,2}\s*[-~]\s*\d{1,2}\b|\bMovies?\s+Collection\b|\bTrilogy\b|\bQuadrilogy\b|\bTetralogy\b|"
+    r"\bAnthology\b|\b\d+\s+Movies?\s+Pack\b|\[\d-\d\]\s*\[m?1080p\]\s*\[\d{4}-\d{4}\]|\bThe\s+Movie\b|"
+    r"\bMovie\s+-\s+00\b|\s-\s*Movie\s*(?:\[|\(|V\d)|\bMovie\s+\d{1,2}\s*-|\bMovie:\s|\bLife\s+is\s+movie\b|"
+    r"\bGekijouban\b|\bFilm\s+Collection\b|\bIl\s+Film\b|\bLe\s+Film\b|\bDer\s+Film\b|\bKolekcja\b|"
+    r"\[YTS\.(?:MX|AG|AM|LT)\]|Criterion#\d+|\bStrong\s+World\s+Episode\s+0\b|\bEpisode\s+\"One\"\b|"
+    r"\bEpisode\s+0\s+-\s+First\s+Contact\b|\bSpecial\.TV\.Collection-\d+\b|\bExtra\s+Edition\b)"
+)
+
+GAME_ROM_VN_RE = re.compile(
+    r"(?i)(?:\bSCUS[\._-]\d+|\bSLUS[\._-]\d+|\bSLES[\._-]\d+|\bNTSC-U\b|\bNTSC-J\b|\[linux-x86\]|"
+    r"\bVisual\s+Novel\b|\bGOG\b|\bFitGirl\b|\bDODI\b|\bsnes-v\.)"
+)
+
+COURSE_BOOK_RE = re.compile(
+    r"(?i)(?:\bTestbuch\b|\bKursbuch\b|\bArbeitsbuch\b|\bUdemy\b|\bCoursera\b|\bPluralsight\b|\bLynda\b|"
+    r"\bData\s+Science\b|\bAssimil\b|\bPimsleur\b|\bFreeCourseWeb\b|\bУроки\s+Фотошоп\b|雅思|IELTS)"
+)
+
+YEAR_RE = re.compile(r"(?<![\dA-Za-z])(?:19[2-9]\d|20[0-2]\d)(?![\dA-Za-z]|\s*[xX×]\s*\d)")
+NOISE_CUT_RE = re.compile(
+    r"(?i)(?:\b2160p\b|\b1080p\b|\b720p\b|\b480p\b|\b576p\b|\b4k\b|\bbluray\b|\bblu-ray\b|\bweb-dl\b|\bwebdl\b|"
+    r"\bwebrip\b|\bbdrip\b|\bdvdrip\b|\bdvd-remux\b|\bdvd\b|\bhdtv\b|\bremux\b|\bx264\b|\bx265\b|\bh264\b|"
+    r"\bh265\b|\bhevc\b|\bav1\b|\bvostfr\b|\bmulti\b|\bdual[\s_-]*audio\b|\beng[\s_-]*sub|\bsub[\s_-]*ita|"
+    r"\bita\b|\bfrench\b|\bgerman\b|\bspanish\b|\brepack\b|\bproper\b|\bremaster\b)"
+)
+
+
+def norm_title(s: str) -> str:
+    """Normalize a title string to NFKC lowercase alphanumeric tokens."""
+    if not s:
+        return ""
+    s = unicodedata.normalize("NFKC", s).casefold()
+    s = re.sub(r"[\W_]+", " ", s, flags=re.UNICODE)
+    return " ".join(s.split())
+
+
+def parse_clean_candidates(raw_name: str, extra_spans: list[str] | None = None) -> tuple[list[tuple[str, int | None]], int | None]:
+    """Extract ordered candidate `(clean_title, year)` pairs sorted longest/most-specific first."""
+    s = raw_name.strip()
+    s = re.sub(r"\.(?:mkv|mp4|avi|m2ts|ts|iso|zip|rar|tar\.bz2|epub|pdf|mp3|flac)$", "", s, flags=re.I)
+    s_no_lead = re.sub(r"^(?:\s*[\[\(][^\]\)]{1,45}[\]\)]\s*|[_\s]+)+", "", s)
+    years = [int(m.group(0)) for m in YEAR_RE.finditer(s_no_lead)]
+    year = years[0] if years else None
+
+    raw_cands: list[tuple[str, int | None]] = []
+
+    def add(ct: str, cy: int | None) -> None:
+        ct = ct.strip()
+        if len(ct) >= 4:
+            raw_cands.append((ct, cy))
+
+    for base in (s_no_lead, s):
+        m_yr = YEAR_RE.search(base)
+        if m_yr and m_yr.start() >= 3:
+            prefix = base[:m_yr.start()]
+            c1 = norm_title(prefix)
+            add(c1, int(m_yr.group(0)))
+            add(c1, None)
+            for subp in re.split(r"(?i)\s+-\s+|\s+aka\s+", prefix):
+                cs = norm_title(subp)
+                if len(cs) >= 5:
+                    add(cs, int(m_yr.group(0)))
+        m_ns = NOISE_CUT_RE.search(base)
+        if m_ns and m_ns.start() >= 3:
+            c2 = norm_title(base[:m_ns.start()])
+            add(c2, year)
+            add(c2, None)
+        c3_raw = re.split(r"[\[\(\|]", base)[0]
+        c3 = norm_title(c3_raw)
+        add(c3, year)
+        add(c3, None)
+        c4 = norm_title(re.sub(r"(?i)(?:\b(?:the\s+movie|movie|ova|oav|special)\b|\s+-\s+\d{1,3}|\s+\d{1,3})\s*$", "", c3_raw))
+        add(c4, year)
+        add(c4, None)
+
+    for grp in re.findall(r"\(([^()]{5,80})\)", s_no_lead) + s_no_lead.split("|")[1:]:
+        if not NOISE_CUT_RE.search(grp) and not YEAR_RE.fullmatch(grp.strip()):
+            for part in re.split(r"[/｜|]", grp):
+                ca = norm_title(part)
+                if len(ca) >= 6:
+                    add(ca, year)
+                    add(ca, None)
+
+    if extra_spans:
+        for sp in sorted(extra_spans, key=lambda x: len(x or ""), reverse=True):
+            nsp = norm_title(sp)
+            if len(nsp) >= 5:
+                add(nsp, year)
+                add(nsp, None)
+
+    raw_cands.sort(key=lambda item: (item[1] is not None, len(item[0])), reverse=True)
+    ordered: list[tuple[str, int | None]] = []
+    seen: set[tuple[str, int | None]] = set()
+    for item in raw_cands:
+        if item not in seen:
+            seen.add(item)
+            ordered.append(item)
+    return ordered, year
+
+
+def lookup_zero_cue_catalog(
+    raw_name: str,
+    catalog: dict[str, Any],
+    is_anime: bool = True,
+    extra_spans: list[str] | None = None,
+) -> tuple[str | None, str | None]:
+    """Query the local validated catalog in strict specificity order:
+    1. `(title, year)` exact match in `title_year_map` (472,038 entries)
+    2. `title` match in `anime_map` when `is_anime=True` and stem length >= 8 (257,008 entries)
+    3. `title` match in unambiguous `title_only_map` (197,739 entries)
+    """
+    cands, yr = parse_clean_candidates(raw_name, extra_spans=extra_spans)
+    title_year_map = catalog.get("title_year_map", {})
+    anime_map = catalog.get("anime_map", {})
+    title_only_map = catalog.get("title_only_map", {})
+
+    for ct, cy in cands:
+        if cy is not None and len(ct) >= 4:
+            key = f"{ct}|{cy}"
+            if key in title_year_map:
+                return title_year_map[key], "title_year"
+
+    if is_anime:
+        for ct, _ in cands:
+            if len(ct) >= 8 and ct in anime_map:
+                return anime_map[ct], "anime_catalog"
+
+    for ct, _ in cands:
+        if len(ct) >= 14 and ct in title_only_map:
+            return title_only_map[ct], "title_only"
+
+    if cands and yr is None:
+        ct0 = cands[0][0]
+        if len(ct0) >= 7 and ct0 in title_only_map:
+            return title_only_map[ct0], "title_only"
+
+    return None, None
+
+
+def apply_structural_and_zero_cue_disambiguation(
+    raw_name: str,
+    probs_category: list[float] | Any,
+    p_adult: float,
+    p_anime: float,
+    catalog: dict[str, Any] | None = None,
+) -> tuple[int, str]:
+    """Apply deterministic structural rules (Stage 1) and Zero-Cue catalog lookup (Stage 2)
+    to an 11-class category probability vector from Armiger or Armiger Ultra.
+
+    Returns `(predicted_category_index, resolution_source)`.
+    """
+    p = [float(x) for x in probs_category]
+    has_tv_cue = bool(EXPLICIT_TV_RE.search(raw_name))
+    has_movie_cue = bool(EXPLICIT_MOVIE_PACK_RE.search(raw_name))
+
+    if has_movie_cue and not has_tv_cue:
+        if p[IDX_MOVIE] + p[IDX_TV] > 0.45:
+            p[IDX_MOVIE] = max(p[IDX_MOVIE], p[IDX_TV] + 0.40)
+            return int(max(range(len(p)), key=p.__getitem__)), "structural_movie_pack"
+    elif has_tv_cue and not has_movie_cue:
+        p[IDX_TV] = max(p[IDX_TV], p[IDX_MOVIE] + 0.40, 0.75)
+        return int(max(range(len(p)), key=p.__getitem__)), "structural_tv_cue"
+
+    if GAME_ROM_VN_RE.search(raw_name) and p_adult < 0.70:
+        p[IDX_GAME] = max(p[IDX_GAME], 0.88)
+        return IDX_GAME, "structural_game_rom_vn"
+
+    if COURSE_BOOK_RE.search(raw_name):
+        p[IDX_COURSE] = max(p[IDX_COURSE], 0.88)
+        return IDX_COURSE, "structural_course_book"
+
+    pred_idx = int(max(range(len(p)), key=p.__getitem__))
+    if catalog is not None and pred_idx in (IDX_MOVIE, IDX_TV) and not has_tv_cue and not has_movie_cue:
+        if max(p[IDX_MOVIE], p[IDX_TV]) < 0.88:
+            cat_hit, src_kind = lookup_zero_cue_catalog(
+                raw_name, catalog, is_anime=bool(p_anime >= 0.20)
+            )
+            if cat_hit in ("movie", "tv_show") and src_kind is not None:
+                return (IDX_MOVIE if cat_hit == "movie" else IDX_TV), f"zero_cue_{src_kind}"
+
+    return pred_idx, "neural_backbone"
+
+
+class ZeroCueCatalogPlugin:
+    """High-level wrapper for loading `validated_zero_cue_catalog.json` and disambiguating predictions."""
+
+    def __init__(self, catalog_path: str | pathlib.Path | None = None) -> None:
+        if catalog_path is None:
+            catalog_path = pathlib.Path(__file__).with_name("validated_zero_cue_catalog.json")
+        self.catalog_path = pathlib.Path(catalog_path)
+        self.catalog: dict[str, Any] = (
+            json.loads(self.catalog_path.read_text()) if self.catalog_path.exists() else {}
+        )
+
+    def resolve(
+        self,
+        raw_name: str,
+        probs_category: list[float] | Any,
+        p_adult: float = 0.0,
+        p_anime: float = 0.0,
+    ) -> dict[str, Any]:
+        idx, source = apply_structural_and_zero_cue_disambiguation(
+            raw_name=raw_name,
+            probs_category=probs_category,
+            p_adult=p_adult,
+            p_anime=p_anime,
+            catalog=self.catalog,
+        )
+        return {
+            "category_index": idx,
+            "category": CATS[idx],
+            "resolution_source": source,
+        }