Commit History
-
Gold-standard hardening: calibration, new solvers, property tests, CI gates
Paul Clark +1: Claude Fable 5Confidence calibration (the honesty flagship): - The heuristic emitted hand-tuned confidences that were badly miscalibrated — a raw 26% meant ~4% real accuracy, a raw 86% meant ~97%. calibrate_confidence() applies an isotonic map (fit on validation by scripts/calibrate_confidence.py) inside heuristic_classify, cutting expected calibration error from 0.18 to 0.04 on the held-out test split. A reported N% now means right ~N% of the time. Two new auto-solvers + a dictionary pass: - beaufort_auto_solve (reciprocal Vigenere, same coordinate-descent refinement) - columnar_auto_solve (recovers column ORDER by hill-climbing the quadgram score; key letters are irrelevant) - Vigenere keyword-dictionary pass for short texts. Benchmarked with keys NOT in the dictionary so the reported statistical rates are honest. Property-based tests (tests/test_properties.py, hypothesis): - decode(encode(x))==x, reciprocity, and generator/core agreement across all reversible ciphers — the class o…77ec8ec -
Overhaul the auto-solvers: full n-gram model, working substitution solver
Paul Clark +1: Claude Fable 5The hill-climbing substitution solver never actually converged (0% success at any length). Root cause was the scoring model, not the search: the old hand-typed ~100-bigram/60-trigram tables were so sparse that their unigram backoff dominated and REWARDED frequency-correct letter salad, making the frequency-seeded start a local optimum of the solver's own metric. - ngram_tables.py (generated by scripts/build_ngram_tables.py): all 676 bigrams, top-3000 trigrams (99.9% coverage), top-5000 quadgrams (99.2%), derived from the corpus' 1.88M letters of English plaintext - english_quadgram_score() with trigram chain-rule backoff; trigram scorer now backs off via the bigram chain rule instead of unigram sums - hill_climb_substitution: Jakobsen-style greedy 325-swap sweeps from the frequency seed + random restarts (default 8), scoring 0.2*bigram + 0.3*trigram + 0.5*quadgram. Verified the true key is the global optimum of the new metric on test texts. - vigenere_auto_solve: coordinat…00a77fa