Commit History
-
feat: char-level inference parity + free-Colab training notebook
Paul Clark +1: Claude Fable 5Makes the --char-level model actually usable in production and gives a zero-cost path to train it. - char_tokenize_text() moved to core.py (pure string op, shared by trainer, app, and evaluator) - train_transformer.py records char_level in the model config so it serializes to config.json and travels to the Hub - app.py and evaluate_baseline.py detect a char-level model from its config and apply the SAME char spacing at inference. Without this a char-level model gets raw text in production and predicts garbage (train/serve skew). - notebooks/train_on_colab.ipynb: open in Colab, pick the free T4 GPU, paste a free HF write token, run — trains the char-level classifier and pushes it to the Hub with no paid GPU Space. DEPLOY.md documents this as the easy path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>dce1610
-
Gold-standard hardening: calibration, new solvers, property tests, CI gates
Paul Clark +1: Claude Fable 5Confidence calibration (the honesty flagship): - The heuristic emitted hand-tuned confidences that were badly miscalibrated — a raw 26% meant ~4% real accuracy, a raw 86% meant ~97%. calibrate_confidence() applies an isotonic map (fit on validation by scripts/calibrate_confidence.py) inside heuristic_classify, cutting expected calibration error from 0.18 to 0.04 on the held-out test split. A reported N% now means right ~N% of the time. Two new auto-solvers + a dictionary pass: - beaufort_auto_solve (reciprocal Vigenere, same coordinate-descent refinement) - columnar_auto_solve (recovers column ORDER by hill-climbing the quadgram score; key letters are irrelevant) - Vigenere keyword-dictionary pass for short texts. Benchmarked with keys NOT in the dictionary so the reported statistical rates are honest. Property-based tests (tests/test_properties.py, hypothesis): - decode(encode(x))==x, reciprocity, and generator/core agreement across all reversible ciphers — the class o…77ec8ec
-
feat(training): add early stopping, cosine LR schedule, and push-to-hub
Paul Clark- EarlyStoppingCallback with --early-stopping-patience (default 3 epochs) prevents overfitting on the tiny museum-only classes (book_cipher: 48, homophonic: 76, etc.) during the 10-epoch focal-loss run. - lr_scheduler_type=cosine: cosine decay outperforms linear for transformer fine-tuning with warmup, especially over 10 epochs. - --push-to-hub / --hub-model-id flags: after training, push the saved model directly to the HF Hub so the Space picks it up via CIPHER_MODEL_ID. - Makefile: added train-push target (train-best + hub push); updated PHONY and help text. Usage: make train-push HUB_MODEL_ID=yourname/cipher-model3ca03a7 -
fix: remove duplicate-tail dead code; harmonise app label vocabulary
Paul Clark +1: Claude Opus 4.7 (1M context)scripts/train_transformer.py had ~160 lines of botched-merge duplicate code after the new main(): a second def main() at line 296 shadowed the new one, and the second `if __name__ == "__main__": main()` invocation ran the OLD trainer right after the new one — silently re-training with different hyperparameters and overwriting the saved model. Removed the duplicate block; --focal-loss, --lr, --warmup-ratio, --label-smoothing, --grad-accum, fp16 now actually take effect. scripts/generate_dataset.py had a similar duplicate tail (~150 lines). No functional bug since the duplicates didn't redefine main(), but it caused 2× generation work and confused the file. Removed it. Also dropped the unused `string` import. app.py Challenge Mode used legacy labels (caesar_rot, columnar, substitution) that don't exist in the trained dataset, causing the generated answer label to disagree with what Detect Mode reported on the same ciphertext. Switched the dropdown and dispatch to the canonical names (c…c231f50 -
feat: expand to 81 ciphers — improved heuristic, 12 new cipher generators, focal-loss training
Paul Clark- core.py: multi-tier heuristic for all 81 ciphers; Tier 1 achieves 100% F1 on 12+ distinctive ciphers (tap_code, morse, ADFGVX, bacon, polybius, etc.); plaintext detection now fires before affine brute-force to avoid misclassification - scripts/generate_dataset.py: 12 new cipher implementations (beaufort, gronsfeld, autokey, trithemius, porta, scytale, double_transposition, rot13, monoalphabetic, stager_route); --append, --per-label, --labels CLI flags added - data/cipher_examples.jsonl: 10 000 new synthetic examples appended (79 922 total) - scripts/train_transformer.py: focal loss trainer, warmup_ratio, label_smoothing, grad_accum, fp16, class-weighted loss on by default, 10 epoch default - Makefile: train-best target with roberta-base + focal loss + optimal hyperparams - tests/test_core.py: ALLOWED_LABELS expanded to 20 synthetic labels; caesar/caesar_rot both accepted76616a8 -
Improve training: weighted loss, blind eval split, roberta-base defaults, 512 tokens
Paul Clark1a9a201 -
Fix singleton class crash: normalise wallis_ciphers typo, guard train_test_split
Paul Clarkb108e4c -
Fix Trainer API: tokenizer -> processing_class (transformers 5.x)
Paul Clark39faac4
-
ADA/mobile UI, pytest pythonpath, train_transformer PyArrow fix, enriched Explain Mode
Paul Clark5642612 -
Initial Cipher Detective AI public foundation
Paul Clarkf90a967