License-screened retrain (v1.3.0): new head trained on the 3,298-document corpus; card numbers updated

README.md

@@ -35,11 +35,12 @@ Developed by the [**FAIR Data Innovations Hub**](https://fairdataihub.org/) at t
 
 | Version | Date | Notes |
 |---------|------|-------|
+| **1.3.0** | 2026-09-21 | License-screened retrain: 84 documents whose source-repository licenses do not permit redistribution were removed, and the head was retrained on the resulting 3,298-document corpus. Held-out accuracy 93.1%. |
 | **1.2.0** | 2026-09-01 | Stacked head: the 512-d text embedding is scored by a stage-one logistic regression whose poster probability becomes a single text-score feature for the final classifier over [text_score + 15 visual + 15 structural] = 31 features. Held-out accuracy 92.9%. Head weights now store a zero first column so softmax reproduces sklearn's sigmoid exactly; earlier heads doubled the logit, which sharpened reported confidences (decisions at 0.5 were unaffected). Flat heads remain loadable by the package. |
 | **1.1.0** | 2026-08-28 | Feature extraction moved off the AGPL-licensed PyMuPDF to pdfplumber (text and structure) and pypdfium2 (page rendering), both permissively licensed, so the whole stack is MIT-compatible. The head was retrained on the re-extracted features; held-out accuracy is 89.2%. |
 | **1.0.0** | 2026-08-18 | Head trained on the human-validated corpus: 3,381 documents labeled by a three-reviewer survey (Krippendorff's alpha 0.79) with blinded adjudication of the 439 contested documents. |
 
-Earlier unversioned weights (April 2026) were trained on heuristically labeled data and are superseded; they remain available in the repository history.
+Earlier unversioned weights (April 2026) were trained on heuristically labeled data and are superseded.
 
 ## Related Models & Tools
 
@@ -48,7 +49,7 @@ Earlier unversioned weights (April 2026) were trained on heuristically labeled d
 | **PosterSentry** | Multimodal poster classifier (this model) | [fairdataihub/poster-sentry](https://huggingface.co/fairdataihub/poster-sentry) |
 | **poster-sentry** | Installable classifier package | [GitHub](https://github.com/fairdataihub/poster-sentry) |
 | **poster-sentry-training** | Training code and replication | [GitHub](https://github.com/fairdataihub/poster-sentry-training) |
-| **poster-sentry-training-data** | Human-validated training dataset (3,381 samples) | [HuggingFace](https://huggingface.co/datasets/fairdataihub/poster-sentry-training-data) |
+| **poster-sentry-training-data** | Human-validated training dataset (3,298 samples) | [HuggingFace](https://huggingface.co/datasets/fairdataihub/poster-sentry-training-data) |
 | **poster-sentry-evaluation-paper-code** | Reproducible analysis for the paper | [GitHub](https://github.com/fairdataihub/poster-sentry-evaluation-paper-code) |
 | **Llama-3.1-8B-Poster-Extraction** | Poster → structured JSON extraction | [fairdataihub/Llama-3.1-8B-Poster-Extraction](https://huggingface.co/fairdataihub/Llama-3.1-8B-Poster-Extraction) |
 | **poster2json** | Python library for poster extraction | [PyPI](https://pypi.org/project/poster2json/) · [Docs](https://fairdataihub.github.io/poster2json/) · [GitHub](https://github.com/fairdataihub/poster2json) |
@@ -101,16 +102,16 @@ Evaluated against human-validated labels (three-reviewer survey with blinded adj
 
 | Metric | Value |
 |--------|-------|
-| **Held-out accuracy** (508 documents) | **92.9%** (95% CI 90.3 to 94.8) |
-| Macro F1 (held-out) | 0.929 |
-| Precision / Recall / F1 (poster) | 0.932 / 0.925 / 0.929 |
-| Precision / Recall / F1 (non-poster) | 0.926 / 0.933 / 0.930 |
-| Nested out-of-fold accuracy (all 3,381 documents, 5-fold) | 94.4% |
-| Panel reliability with the model as a fourth rater | alpha 0.793 -> 0.797 (unchanged) |
-| Calibration (out-of-fold) | Brier 0.045, ECE 0.017 |
+| **Held-out accuracy** (495 documents) | **93.1%** (95% CI 90.6 to 95.0) |
+| Macro F1 (held-out) | 0.931 |
+| Precision / Recall / F1 (poster) | 0.918 / 0.948 / 0.933 |
+| Precision / Recall / F1 (non-poster) | 0.946 / 0.915 / 0.930 |
+| Nested out-of-fold accuracy (all 3,298 documents, 5-fold) | 93.9% |
+| Panel reliability with the model as a fourth rater | alpha 0.792 -> 0.795 (unchanged) |
+| Calibration (out-of-fold) | Brier 0.046, ECE 0.016 |
 | Classifier speed | under one second per file on a CPU (PDF parsing is the bottleneck) |
 
-Errors concentrate where the human panel itself divided: out-of-fold agreement is 96.5% on documents the panel rated unanimously and 81.1% on documents decided two to one.
+Errors concentrate where the human panel itself divided: out-of-fold agreement is 96.4% on documents the panel rated unanimously and 77.3% on documents decided two to one.
 
 ### Top Features by Importance
 
@@ -118,29 +119,29 @@ Standardized logistic regression coefficients of the trained head (positive push
 
 | Rank | Feature | Coefficient | Signal |
 |------|---------|------------|--------|
-| 1 | `page_count` | -3.87 | More pages pushes away from poster |
-| 2 | `size_per_page_kb` | +2.49 | Dense, high-resolution single pages |
-| 3 | `line_count` | +2.00 | Posters pack many short text lines |
-| 4 | `file_size_kb` | -1.74 | Multi-page documents are bigger overall |
-| 5 | `mean_g` | +1.30 | Colorful, non-white pages |
-| 6 | `img_width` | +1.21 | Large rendered width |
-| 7 | `page_width_pt` | +1.16 | Posters are physically wide |
-| 8 | `edge_density` | +0.83 | Visually busy layouts |
-| 9 | `is_landscape` | +0.75 | Many posters are landscape |
-| 10 | `color_diversity` | +0.75 | Posters are visually rich |
-
-Every coefficient of the final classifier is a named feature; the whole text channel enters as the single `text_score` (+0.64, rank 12 of 31), whose value concentrates on the documents where geometry misleads.
+| 1 | `page_count` | -3.26 | More pages pushes away from poster |
+| 2 | `size_per_page_kb` | +2.42 | Dense, high-resolution single pages |
+| 3 | `line_count` | +1.94 | Posters pack many short text lines |
+| 4 | `file_size_kb` | -1.70 | Multi-page documents are bigger overall |
+| 5 | `mean_g` | +1.10 | Colorful, non-white pages |
+| 6 | `is_landscape` | +1.00 | Many posters are landscape |
+| 7 | `white_space_ratio` | -0.95 | Sparse, white pages push toward non-poster |
+| 8 | `edge_density` | +0.86 | Visually busy layouts |
+| 9 | `page_width_pt` | +0.80 | Posters are physically wide |
+| 10 | `img_width` | +0.79 | Large rendered width |
+
+Every coefficient of the final classifier is a named feature; the whole text channel enters as the single `text_score` (+0.68, rank 13 of 31), whose value concentrates on the documents where geometry misleads.
 
 ## Training Data
 
-Trained on **3,381 documents with human-validated labels**, zero synthetic data:
+Trained on **3,298 documents with human-validated labels**, zero synthetic data:
 
 | Class | Count | Label provenance |
 |-------|-------|------------------|
-| **Poster** | 1,686 | Three-reviewer survey; unanimous panel label or blinded adjudication |
-| **Non-poster** | 1,695 | Three-reviewer survey; unanimous panel label or blinded adjudication |
+| **Poster** | 1,651 | Three-reviewer survey; unanimous panel label or blinded adjudication |
+| **Non-poster** | 1,647 | Three-reviewer survey; unanimous panel label or blinded adjudication |
 
-Three reviewers independently rated all 3,570 candidate documents (inter-rater Krippendorff's alpha 0.79); the 439 documents without a unanimous panel were settled in a blinded adjudication review. After removing 182 near-duplicate documents and 7 with unavailable PDFs, the remaining 3,381 form the training corpus. Applied to the full corpus of 30,139 readable repository PDFs labeled as posters, PosterSentry classifies 80.6% as posters: roughly one in five records labeled as posters is something else.
+Three reviewers independently rated 3,486 candidate documents that passed license screening (inter-rater Krippendorff's alpha 0.79); the 430 documents without a unanimous panel were settled in a blinded adjudication review. After removing 181 near-duplicate documents and 7 with unavailable PDFs, the remaining 3,298 form the training corpus. Applied to the full corpus of 30,139 readable repository PDFs labeled as posters, PosterSentry classifies 80.5% as posters: roughly one in five records labeled as posters is something else.
 
 Training data: [fairdataihub/poster-sentry-training-data](https://huggingface.co/datasets/fairdataihub/poster-sentry-training-data)
 
@@ -211,7 +212,7 @@ Training completes in ~40 minutes on CPU (PDF rendering is the bottleneck, not t
   title = {PosterSentry: Multimodal Scientific Poster Classifier},
   author = {O'Neill, Jamey and Portillo, Dorian and Zeinali, Nahid and Soundarajan, Sanjay and Blake, Gerard and Sarin, Parth and Buttrick, Adam and Patel, Bhavesh},
   year = {2026},
-  version = {1.1.0},
+  version = {1.3.0},
   url = {https://huggingface.co/fairdataihub/poster-sentry},
   note = {Part of the posters.science initiative}
 }

models/poster_sentry_head.npz

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3ba0a0bb45d65cffc622ed00881c1c45b3080636b6338b012d1d1b0a802cfd25
+oid sha256:d23af6c5235bff4e1521261fc615933c119880c6cd13f46fa53d30af69d832d7
 size 20002