Backtest
Generated by python scripts/backtest.py on 2026-10-19. Each prediction
uses only the history a user had before that contest, built by the same feature
code the API serves, and is compared with the rating change LeetCode applied.
| Metric | Value |
|---|---|
| Contests scored | 173 |
| Users | 50 |
| MAE | 9.11 |
| RMSE | 19.85 |
| Median absolute error | 5.59 |
| Predictions within 10 points | 73.4% |
| Direction right (gain vs loss) | 90.8% |
| MAE of always predicting 0 | 29.34 |
Users were sampled with seed 7 from data/usernames.json past index
12000, the slice scripts/update_data.py never fetches, so none of
these contests is in the training data. Up to the last 5 rated contests
per user count, if they started within 365 days of the run.
869 sampled accounts had no contest in that window.
Model: models/weights.npz.