# Backtest

Generated by `python scripts/backtest.py` on 2026-10-19. Each prediction
uses only the history a user had before that contest, built by the same feature
code the API serves, and is compared with the rating change LeetCode applied.

| Metric | Value |
|---|---|
| Contests scored | 173 |
| Users | 50 |
| MAE | **9.11** |
| RMSE | 19.85 |
| Median absolute error | 5.59 |
| Predictions within 10 points | 73.4% |
| Direction right (gain vs loss) | 90.8% |
| MAE of always predicting 0 | 29.34 |

Users were sampled with seed 7 from `data/usernames.json` past index
12000, the slice `scripts/update_data.py` never fetches, so none of
these contests is in the training data. Up to the last 5 rated contests
per user count, if they started within 365 days of the run.
869 sampled accounts had no contest in that window.
Model: `models/weights.npz`.