Upload docs/en/EVALUATION.md with huggingface_hub

docs/en/EVALUATION.md

@@ -0,0 +1,59 @@
+# Evaluation
+
+This document explains how the **87.6% login success rate** was measured, so the result is reproducible and trustworthy.
+
+## Core principle
+
+There is no public ground-truth dataset for this captcha, so we use **real forum logins** as the sole, 100%-reliable judge:
+
+> **"Login success" from the forum = the model was correct**
+> **"Code incorrect or expired" = the model was wrong**
+
+Every attempt is a clean binary verdict — no subjective estimation.
+
+## Test flow (per attempt)
+
+1. Fetch the login page → parse the form and captcha URL (with session cookie)
+2. Fetch the captcha image (timestamp-cache-busted, 150x60 PNG)
+3. Model inference → 4 digits
+4. Submit login (session cookie + hidden form fields + the code)
+5. Read the forum response → success / "code incorrect"
+6. Clear the local session → next attempt starts as a fresh guest
+
+Guarantees:
+- Each captcha is bound to its own session; recognition and submission are 1:1, no session mismatch
+- Failed responses are dumped via `SP_DEBUG_LOGIN` and checked one by one to confirm genuine "code incorrect", not network/rate-limit pages
+- Network-level interference is excluded from statistics
+
+## Test environment
+
+| Item | Value |
+|---|---|
+| Site | south-plus.net |
+| Network | home IP + Cloudflare IP pinning (bypasses DNS poisoning) |
+| Request interval | 7 seconds (respecting forum rate limits) |
+| Account | a real, valid account |
+| Model | captcha_1000_raw.onnx (88.6% val) |
+
+## Results
+
+**100 logins (final formal test)**
+
+| Result | Count |
+|---|---|
+| Login success | **85** |
+| Code incorrect | 12 |
+| Rate-limited (excluded) | 3 |
+| Valid total | **97** |
+
+**Per-attempt success = 85/97 = 87.6%**
+
+Early small sample: 13/15 = 86.7% (for reference).
+
+**With retries**: `P(success within 6 tries) = 1 - (1 - 0.876)^6 ≈ 99.9997%` — about 3 in 10,000 logins fail after 6 retries.
+
+## Confidence
+
+- 100-attempt sample with hard binary verdicts → high statistical confidence
+- Consistent with the 15-attempt sample (86.7%) → stable result
+- Held-out validation set (79 images) at 88.6% cross-confirms the login figure (87.6%)